VMware Horizon Community
VUTiger
Enthusiast
Enthusiast

App Volumes multi-Data center design?

We are deploying Horizon View between two datacenters about 30miles apart.  They are interconnected with a dual 10Gb vpc link so our roundtrip latency is only 2ms.

As this is a greenfield install we are starting redundant across the board, two internal connection servers per site, 2 security servers with thier own connection servers per site.

We do not have vplex or stretched cluster but with load balancing and multi-site availability we can build it so everything stays up and users can connect to either datacenter and continue to work.

App Volumes is a newly added feature we are loving, however managing muli-site with this is interesting.  As of 2.5 we were planning on 2 app volume servers per site connection to external SQL DB and just deploying the app stacks in pairs between datacenters.

However in the new version (2.6) they add multi-vcenter support as well as storage groups.

We are trying to determine if 1 pair of app volumes with DB/host failover would suffice or if the multi-vcenter is targeted at internal site pods and ability to expand sites with multi-vcenters at one datacenter.

Anyone know or have some thoughts to weigh in on this? 

With one central app volumes and ability to add multi-vcenter to it and storage groups we could target app stacks to specific vcenters storage and attach accordingly (App-stack update and name it Primary-DC1 and do another update and call it Primary-DC2 and go from there from one central location)

Are we adding a single point of failure that requires a "manual Fail over" where cloud pod and active-active provide everything else to be online regardless of site failure.

Thanks for any help, ideas or insights!

27 Replies
vManiaEK
Contributor
Contributor

Hi, we are also building a similar multi-data center solution and want to use AppVolumes and have fully automated failover of all components.

Also wondering how we should keep our AppStacks and Writeable volumes in sync between data centers ?

Can such designs support 20k+ users?

Thanks

Reply
0 Kudos
Ray_handels
Virtuoso
Virtuoso

He,

Also would like to chip in on the discussion.

We made some quite serious plans and actually gave it some thoughts Smiley Happy Smiley Happy

We now have 2 different sites with 2 different "building blocks" with a SAN on both sides and a Appvolumes Manager on both sites. The managers both point to one database so if on of the ESXi hosts fails were the AppVolumes manager is on all will still work.

We also added multiple storages into the manager. What you need to do is change the default storage location of the appstacks (go to configuration Storage and choose both or more storages) and then click upload prepackaged volumes. If you do this AppVolumes will see multiple datastores and use both.

My understanding is if you have multiple sites and VDI1 runs on site 1 and VDI2 runs on site 2 that the appstacks of site 1 will be used for VDI1 and the appstacks on site 2 will be used on site 2. AppVolumes checks were the VDI resides and attaches the appstacks on that site.

Regarding writable volumes we are looking at storage sync. We would like to create half of the writable volumes on site 1, half on site 2 and let the entire cloudvolumes\writables directory be replicated to the other site and vise versa.

If anyone has any better ideas of findings or whatever, please feel free to chip in as we are still looking at the perfect solution Smiley Happy

Reply
0 Kudos
matthiasFF
Enthusiast
Enthusiast

We are running two sites based on the VMware View AlwaysOn Design guide and are in the process of implementing AppVolumes.

After installing version 2.6 of the Appvolumes Manager, i can't figure out how to add multiple vCenters? Is this a hidden/experimental feature?

Would running two instances of Appvolumes Manager, with different vCenter servers (siteA/siteB), accessing the same SQL database work after one site goes down? Of course i'd have multiple mgmt servers added on the golden image.

Thanks

Reply
0 Kudos
Ray_handels
Virtuoso
Virtuoso

For as far as i know you cannot use multiple VCenter servers with appvolumes.

It is not possible to use 1 database with Appvolumes if they both connect to a different VCenter server. Problem is that the connection setting to Vcenter is also in the database.

Good thing about Appvolumes is that it is extremely easy to configure 2 Appvolumes servers with seperate databases without to much of a hassle.

Only thing you need to do is create Appstacks and writable volumes in 1 place and then let both Appvolumes managers either create or import the appstacks as they are created.

Still. If a user can log into both VCenter servers and have a desktop available do keep in mind that when the writable volume is attached to machine A the user cannot log in anymore because he or she needs to have exclusive permissions to the writable volume.

You also need to deal with licenses if you use 2 databases, that might also be an issue.

Reply
0 Kudos
matthiasFF
Enthusiast
Enthusiast

Thank you so much for clarifying and sorry for hijacking the thread.

My gut says licensing will be a problem. :smileyplain:

Reply
0 Kudos
VUTiger
Enthusiast
Enthusiast

Actually you can.

If you Edit the Hypervisor Tab under Configuration it gives you Host name to use for your vCenter.  Simply insert a comma after your first vCenter and enter your second.  It does require your ESX and vCenter service accounts have same password for both sites.

Once it re-scans you can see all storage at both sides.

From there simply create storage groups with both sites App-Vols read only storage and ensure you set the automation option to replicate appstacks.

You now have one instance replicating to both vcenters storage.

From here you create a VIP on your load balancer to hit view-appvols which points to all your App Vols Managers.

You can also add multiple managers to the App Vol Agent.  This way you target your main site collection of App Vol Managers first and leverage the multiple entries to add the other site App Vol managers second, third or forth.

Now to solve one DB to rule them all for App Vols, I would suggest Availability groups with MS SQL and another VIP that your App Vol Managers should point to that hits the IP of the DB on both sites.

This way if one site goes down, the Availability group in SQL has the other site online and your manager should be hitting it via the VIP.

Questions?

Realized I answered my own thread.  SQL Availability groups have been a recent feature we have added but should work.  That tied with recent KB below substantiates my initial question.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=209183...

JHT_Seattle
Hot Shot
Hot Shot

This was a really well done, informative post.  Thank you!

We're configuring similarly, and are in the process of setting up SQL Always On now.  I'll sleep better at night after having Manager and SQL redundancy before moving all of our users to an App Volumes delivery environment.  We're going to have two View pods in the same datacenter, so the improvements to App Volumes as of 2.6 to be able to use multiple vCenters and storage groups solves a pretty big challenge for us.

Now, if I could only update to 2.6... Smiley Wink

Bring on 2.7!

Reply
0 Kudos
gv74
Enthusiast
Enthusiast

I was intrigued by this undocumented feature, as it may have solved a headache I currently have with AppStack replication.  Unfortunately, although you can do exactly what VUTiger describes (right down to creating Storage Groups) App Volumes does not allow replication to occur between vCenters.  An error to that effect is posted in the System Messages tab, as attached.

This is a shame, as in my environment I have two production App Volumes installations in our London & Singapore datacenters respectively.  Although the installations are completely independent (separate Managers & SQL servers etc.), we wish to deploy AppStacks globally to ensure consistency across sites.  Replicating the AppStacks is therefore a challenge - especially keeping them thin-provisioned.  In fact, I have found the 'quickest' way to do it is simply have a provisioning VM both sides & install the app simultaneously into each.  Although this works it is a duplication of effort - both in terms of installing the apps themselves & updating them thereafter (plus maintaining the provisioning VMs).

Given these two sites will remain separate, it would be useful to have some kind of 'blind replication' facility within the Manager, whereby AppStacks can replicate across vCenters but do no more than that.  I would then manually import them the other end once replication has finished.  Neither Manager would be required to know an identical AppStack exists somewhere else - I just want a quick & easy replication facility!

I was talking to one of our solution partners the other day about this (and other App Volumes feature requests).  He strongly hinted that major feature enhancements were coming this year but could say no more because he was under NDA.  Whether or not it will include AppStack consistency across sites who know but fingers crossed!

I do believe App Volumes is a revolutionary product but there is clearly still much more to come.  Right now we are just sticking to the core functionality (i.e. provisioning & presenting AppStacks) which works very well for us - well enough that we now have over a dozen stacks presented to several hundred users across our two datacentrers.  As it stands the documentation is sparse at best & features like Storage Groups feel unrealised but hint at much richer functionality to come.  I'm very much looking forward to seeing how the product matures moving forwards.

JHT_Seattle
Hot Shot
Hot Shot

I suspect that some replication functionality could be achieved via PowerCLI scripting, since it should just be a matter of copying files from one datastore to another.  You could probably create a script to compare the two locations and then copy whatever has changed from your "master" datastore.  Once PowerCLI catches up to App Volumes (or vice-versa?) you could have the stack import function automated as well.

That, or store the stacks on some share with DFS replication and save yourself some trouble.  I'm not sure how fast that would be given your locations, but you'd know your network better than anyone else Smiley Happy

Having said all of that, we had issues with some stacks after copying them between datastores and importing them into a new Manager.  No rhyme or reason to the ones that failed, but reimporting made no difference.  Our only way forward was to recreate the stack in the 2nd environment.  Bit of a drag.

Reply
0 Kudos
VUTiger
Enthusiast
Enthusiast

Sure enough, theory vs reality are far different.

Would have been perfect if it would have replicated across both sites storage as it has access via both vcenters and esxi hosts.

Since our Datacenters are so close, we are leveraging the 1DB-many App Volumes Manager servers (4, 2 per datacenter).

Similar to GV74 we are creating app stacks across the independent storage but when we will build an App Stack we will do it twice, provision to two independent provisioning servers, install in unison and call it good.

Now all app stacks are in one site (Replicated database) just still requires individual updates in duplicate and creation.

Looking forward to what they bring to help enable multi-site configuration for this product.

gv74
Enthusiast
Enthusiast

Good to hear you can leverage that functionality now VUTiger - must be a time-saver for you.  Even though the Storage Groups feature feels a little 'work in progress' right now, I can still see it being very useful if you can work in a single DB environment.  Unfortunately for me with a RTD between London & Singapore of over 180ms, I can't even do hardware replication on our 3PAR - let alone entertain the prospect of a 1 DB to many Manager architecture!  Never mind, here's hoping VMware will bring something to the table that might help.

For now I'll carry on provisioning both sides. Yes it's a duplication of work & we have looked at scripted alternatives but I always seem to come back to spinning both those provisioning VMs up in the end. Gets the job done!

Reply
0 Kudos
matthiasFF
Enthusiast
Enthusiast

Does anyone have any thoughts on the following scenario? I have a few questions as i have been testing the last couple of days.

Note: Although they are seperate sites, they are connected via 4x 10Gbit darkfibre links. Two switch stacks (1 for iscsi/1 for the rest) with one active and one passive link per stack.

We run a Horizon View 6.0.1 environment with both sites active, but enough ressources to run everything at one site (DR).

- Is it possible/supported to set up multiple ODBC system-DSNs for each Manager instance (automatic failover) like with vCenter and Connection servers?

- When using storage groups and having appstacks on multiple datastores and one of the datastores "goes down", will it automatically attach another one to the VM?

- How are datastores in the storage group assigned to VMs? Are they distributed evenly? The distribution strategy can only be set for writeable volumes...

I'm sorry if these are stupid questions... The user guide is the only documentation i found and it's not very helpful so i'm asking here.

mf_Appvol_Konzept_inet.png

Thank you very much.

Reply
0 Kudos
snapmatt
Contributor
Contributor

Using SQL replication between two databases of two App Volumes Managers deployments is possible (VMware is producing a document to describe how to do this), you would mostly want to replication the snapvol* tables and the user/computer/groups tables but not the svconfiguration table since that would be unique to each location.

> Is it possible/supported to set up multiple ODBC system-DSNs for each Manager instance (automatic failover) like with vCenter and Connection servers?

Not multiple ODBC DSNs but App Volumes Manager is using SQL Server Native Client for ODBC so you can make this highly available within a single ODBC DSN this like this: SQL Server Native Client Support for High Availability, Disaster Recovery

> When using storage groups and having appstacks on multiple datastores and one of the datastores "goes down", will it automatically attach another one to the VM?

At the time of login where the volume would be attached, it should exclude from consideration any inaccessible datastores (this can be manually tested by just disabling/disconnecting all of the datastores in the storage group but one and make sure the volume still gets attached)

> How are datastores in the storage group assigned to VMs? Are they distributed evenly? The distribution strategy can only be set for writeable volumes...

The one with the least attachments is used. This is so that if you add a new datastore to a storage group, all the attachments will be from that datastore until it reaches the same level of attachments as the existing datastores.

Reply
0 Kudos
elgwhoppo
Hot Shot
Hot Shot

gv74, with which version of the AppVolumes did you experience the attached error message with?


I am wondering if with the new native multi-vCenter capabilities in 2.9 you would still be prevented from having a storage group replicate between two vCenter servers.

VCDX-Desktop
matthiasFF
Enthusiast
Enthusiast

Wow, huge coincidence.

I just tried replicating between datastores on different vCenter servers like 20 minutes ago, and it still doesn't work in v2.9.

2015-06-24 14_27_48-App Volumes Manager.png

Here is what's officially supported: VMware App Volumes Multi-vCenter and Multi-Site Deployments - VMware Blogs

elgwhoppo
Hot Shot
Hot Shot

Thanks for the reply matthias. For now I'm going to try some kind of copy script that can be run in batch.

For those wondering, I've drawn up a couple possible solutions and highlighted them here. AppVolumes 2.9 – Near 0 RTO Multi-Datacenter Design Options | elgwhoppo's vNotebook

VCDX-Desktop
Reply
0 Kudos
matthiasFF
Enthusiast
Enthusiast

That was a great read, I'm in an almost identical situation but with a smaller userbase. We also have more than enouph bandwidth between sites.

What I have done now is basically create two storage groups, one for each vCenter. In each storage group are 2 datastores only vcenter A can access, and a third "transfer" datastore (accessed through vcenter A). The transfer datastore is attached to all hosts at each site and accessible from both vcenter servers. With automatic replication the .vmdks are replicated to each datastore at both sites. The "downside" is some linked clones from one site might use an appstack from the other site, but like i said bandwidth and latency isn't a problem for us.

A bad visio drawing to show what i meant:

2015-06-26 16_25_18-mf_Appvol_Konzept_inet.vsdx - Visio Professional.png

Just food for thought Smiley Happy

elgwhoppo
Hot Shot
Hot Shot

That's a great idea. So each side has a storage group which includes the transfer LUN, which ensures that everything is replicated. Depends on getting native storage connectivity between the datacenters rather than just network, but still, that's a pretty great workaround.

Do you find that everything replicates across all the datastores in a consistent fashion when updates are initiated from one side?

Also, are you leveraging that just for writables or for writables and appstacks?

VCDX-Desktop
Reply
0 Kudos
matthiasFF
Enthusiast
Enthusiast

Exactly.

Yes, they replicate consistently! With automatic replication and import everything works just fine at both sites. I believe the intervall for replication is about 1h default. Assignments still need to be done manually of course.

Worth mentioning is we only have around 30GB of appstacks though...

We are using only appstacks.

Next week i will be testing the automatic failover on our SQL mirror (still using 2008R2). If that works we are almost good to go into production.

Reply
0 Kudos