mrstorey303 Tracker

Re: vSAN shutdown cluster Powercli function

mrstorey303 — Fri, 15 Sep 2023 08:29:29 GMT

+1 to this - we actually have a support case open with VMware because we can't find what permissions are required to use the 'shutdown vsan' and 'restart / startup vsan' cluster other than top level / administrator@vsphere.local privs.

We need to scope a role for tenants within our shared vcenter to do this - but it doesn't seem to be documented anywhere. lmk if you have any insight!

Thanks

Re: NSX Application Platform Automation

mrstorey303 — Tue, 05 Sep 2023 10:38:34 GMT

Fwiw I heard back from VMware about this.

In my case, this is because I'm using a very recent (8.1+) version of vcenter, which deprecates tkg release 1.21.6, which causes an issue for 0.2.x versions of the appliance.

As I suspected, there is a new release of the automation appliance coming out soon which will address this.

Re: NSX Application Platform Automation

mrstorey303 — Mon, 04 Sep 2023 16:07:07 GMT

+1 - I'm getting the same here.

To me this reads like the automation appliance is pre-loaded with whatever yaml deploys tkg clusters, and it's trying to deploy a deprecated version.

Basically, we prob need a new version of the OVA.

Will reach out to support + account team unless you have heard anything?

NSX-T Identity Firewall Event Log Scraping - Large Environments

mrstorey303 — Wed, 25 Jan 2023 00:07:58 GMT

Is it possible to configure Identity Firewall Event Log Scraping to servers that hold subscribed events? Or do event log servers in the id firewall configuration *have* to be domain controllers, because there's no way of telling NSX to look in an event log other than the security log?

We have a pretty large environment - NSX-T 3.2.2, multiple domains, many sites, many domain controllers. Since domain controllers only hold events for logon attempts against that particular domain controller, we'd end up having to configure a lot of event log servers in NSX.

With other user ID solutions we've been able to configure event log forwarding on a box, which acts as an aggregation point for these types of events, and we point the solution at that. Is it wishful thinking I can get NSX-T to do that?

Thanks in advance

NSX Application Platform Without Tanzu?

mrstorey303 — Mon, 12 Sep 2022 17:56:34 GMT

Has anyone has success deploying the NSX Application Platform on a k8s platform that *is not* Tanzu?

Documentation says that it's supported on 1.17 - 1.21 upstream k8s compliant cluster:

https://docs.vmware.com/en/VMware-NSX-T-Data-Center/3.2/nsx-application-platform/GUID-D54C1B87-8EF3-45B3-AB27-EFE90A154DD3.html

Ideally we'd like to deploy NAPP on a managed k8s instance, such as EKS or GKE, but to us so far, it seems extremely difficult to deploy on anything that is not Tanzu..

Deploying on EKS for example, will throw an error saying there aren't enough control nodes for advanced deployments (ie requires 3, but EKS doesn't expose control node counts because the mgmt plane is a managed service).

NSX Application Platform Deployment Failed - 'Registration Failed'

mrstorey303 — Tue, 12 Jul 2022 10:32:52 GMT

I’m struggling to deploy the NSX Application Platform in our environment. The deployment prechecks all pass, but the deployment consistently fails at the ‘Registering Platform’ step. Admittedly I am a newbie when it comes to Tanzu and k8s, but hoping someone can point me in the right direction.

I have deployed a Tanzu CE cluster, 3 control plane nodes and 3 worker nodes. All meeting the spec required to deploy NSX intelligence (16CPUs, 64GB RAM, 1TB Disk). Kube VIP + antrea is used for networking.

MetalLB has been configured to provide an entry point for the service name / fqdn. It has been given a pool of 15 addresses, and I have configured 2 A records to point to the first two addresses from this range:

Service name - nsx-application-platform.domain.com

Messaging Service Name - nsx-application-platform-msn.domain.com

(To be honest, I’m not exactly clear what the ‘messaging service name’ is - it seems new with nsx 3.2.x - I’m also just taking it on faith that the deployment will somehow assign the correct IPs from the metallb pool, to correspond with the A records I have created…..)

For context, I’ve been using this chap’s guide, and found it very helpful - https://lumberjackwizard.com/2022/03/09/deploying-nsx-application-platform-part-six-metallb/

Aside from the obvious symptom / error of ‘NSX Application Platform Registration failed’ during deployment, the only other errors I can see are these, which occur on the metallb speaker pods

Events:

Type Reason Age From Message

---- ------ ---- ---- -------

Warning Unhealthy 45m kubelet Liveness probe failed: Get "http://10.50.16.169:7472/metrics": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

Warning Unhealthy 45m kubelet Readiness probe failed: Get "http://10.50.16.169:7472/metrics": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

That said, all the pods in the metallb namespace look like they are running ok:

>kubectl get pods -n metallb-system

NAME READY STATUS RESTARTS AGE

controller-66445f859d-589zw 1/1 Running 0 20h

speaker-c6gqt 1/1 Running 0 20h

speaker-dnrbh 1/1 Running 0 20h

speaker-ncpcl 1/1 Running 0 20h

speaker-qg6zz 1/1 Running 0 20h

speaker-qt7mw 1/1 Running 0 20h

speaker-r6kgs 1/1 Running 0 20h

I appreciate these scenarios are very difficult to diagnose and troubleshoot - but I’d really appreciate any pointers you could throw my way!

Thanks in advance

vLCM & Auto Deploy - Stateful Install to Remote / SAN Disk

mrstorey303 — Thu, 30 Sep 2021 09:22:05 GMT

I'm trying to use auto deploy to perform stateful installs on diskless ESXi hosts to an FC SAN disk on our Flasharray (using the 'remote' disk argument).

This works fine unless I configure auto deploy to build the host into a vLCM enabled cluster (which annoyingly is what I want...)

When using a vLCM cluster, auto deploy always fails with 'install --firstdisk specified, but no suitable disk is found' during install. It seems that regardless of what disk arguments you set in the host profile associated with the vLCM cluster, auto deploy insists on using 'firstdisk' - when I want it to use 'remote'.

It works fine if I deploy the host outside the cluster - so, I guess I have a workaround (I could just move the hosts into the cluster after they are deployed), but it feels like this is an undocumented feature / limitation of vLCM and auto deploy, unless I'm missing something.

Anyone with similar experiences?

Re: Horizon 8.2 / F5 iApp + Connection Server HTML5 Tunnelling

mrstorey303 — Wed, 08 Sep 2021 16:08:57 GMT

To further this - we're also finding that having x-forwarded-for enabled on the our F5 also prevents DEM smart policies from being able to detect the gateway location (ie internal / external).

This is preventing policy based cut / paste operations from working, ie we want 'internal' users to have inbound+outbound cut and paste, and disabled for external.

Anyone else seeing this?

Re: Horizon 8.2 / F5 iApp + Connection Server HTML5 Tunnelling

mrstorey303 — Fri, 13 Aug 2021 17:03:52 GMT

Reply from F5 support:

re: x-forwarded-for:

- This is an application requirement and depends on the application itself. The BigIP unit only inserts the client's IP address as a header for the application to be aware of source of connectivity and requests.

re: why would disabling x-forward-for make a difference anyway?

- This is something that VMWare team will have to explain. Browsers identify themselves with "agent data", so application should have no problems identifying them as such.

re: are F5 going to fix / update the iApp

- If there is an issue with new Horizon version that needs to be fixed, VMWare will provide a fix/patch/workaround.

@wing523 - do you have any detail on why disabling x-forwarded-for is actually required for selective tunnelling to work on recent horizon versions? Appears there is a bit of finger-pointed regarding why the issue occurs, and who's responsibility it is to fix.

Thanks

Re: Horizon 8.2 / F5 iApp + Connection Server HTML5 Tunnelling

mrstorey303 — Mon, 09 Aug 2021 14:55:38 GMT

Reopening this for discussion if possible - while I follow up with F5 re: iApp configuration - anyone know from within VMware why disabling this feature would allow the connection servers to start correctly / selectively tunnelling for html5 again? And why this is a change in Horizon 8.x ?

Not an expert, but I all I thought x-forwarded-for did was just send the client IP to the web server, for logging / auditing purposes etc. Why would the connection servers break tunnelling if F5 sent this info?

Re: Horizon 8.2 / F5 iApp + Connection Server HTML5 Tunnelling

mrstorey303 — Tue, 03 Aug 2021 07:39:36 GMT

My bad! Looks like someone disabled tunnelling on one of the connection servers in the pair in between testing.

Just went back to double check the config and it works!

Thanks to all involved. Will reach out to F5 to see if they'll consider updating their iApp to reflect these changes - doesn't look like it's been updated in a while.

Re: Horizon 8.2 / F5 iApp + Connection Server HTML5 Tunnelling

mrstorey303 — Mon, 02 Aug 2021 09:55:51 GMT

Sorry for the delay with this - I retried disabling this on the http client profile (the one the iApp creates), and also setting the client profile to 'none' on the LTM, but I still get the cert warnings.

Trying to follow up with F5 support now - will report back if I find anything useful.

Re: Horizon 8.2 / F5 iApp + Connection Server HTML5 Tunnelling

mrstorey303 — Mon, 05 Jul 2021 16:20:44 GMT

Thanks for your reply - I'm aware of the KB but that config is something I'd like to avoid - using wildcard certificates for desktops and configuring view to access agents via DNS name isn't really palatable - I'd just like to get 8.2 to behave in the same way 7.12 was in terms of connection server html5 tunnelling and F5 LTMs.

I saw the release notes, and admittedly reading 'the forwarding rules for HTTP requests.....have changed..' certainly seems relevant, but the frontServiceWhitelist config just disables the admin console on a connection server, it doesn't seem related to html5 blast tunnelling.

Horizon 8.2 / F5 iApp + Connection Server HTML5 Tunnelling

mrstorey303 — Mon, 05 Jul 2021 13:52:57 GMT

Anyone out there using the Horizon F5 iApp to front multiple connection servers?

Since upgrading from Horizon 7.12 to Horizon 8.2 our F5’s don’t seem to work with selective tunnelling to HTML5 clients anymore, so this naturally throws a cert warning because the URL offered is that of the remote desktop IP rather than the blast external URL (so, basically not tunnelled).

We’re running the v1.5.9 of the Horizon iApp, and we have “Use Blast Secure Gateway for only HTML Access Connections to machine” set on the connection servers. The iApp has ‘No, Blast connections should not go through the BIG-IP system’. But I’ve also tried with ‘Yes - blast should go through BIG-IP + Yes, Blast proxied by UAGs’ which has the same non-tunnelling result.

External HTML5 client connections via a UAG work fine (and associated F5 LTM), Not sure what’s changed since the upgrade re: selective HTML5 tunnelling for load balanced connection servers.

Anyone with similar setups experienced this?

Thanks in advance

Large Linked Mode Environments - Backup, Recovery, Operations

mrstorey303 — Mon, 28 Sep 2020 10:51:02 GMT

Its looking like I need to break up our centralised, multi-tenant vCenter model up into a individual vCenters per tenant, pretty much because of the limitations of NSX - ie not able to scope access for distributed firewall admins to 'per tenant' ESX hosts (I want to prevent a tenant from pushing firewall rules to other tenant's esxi nodes).

Splitting up the vcenters I'm fine with - in many ways it'd make my life simpler. But I'm getting pressure internally to consider deploying all of them into a single SSO domain - and given my recent (bad) vCenter upgrade experiences, and the rollback / DR prep you need to do in order to recover from a failed upgrade when using linked mode, it fills me with dread.

I guess you're all aware - the only supported rollback method (outside of recovering from file based vcsa backups) is to:

- Power down ALL vcenters in the SSO domain (or at least stop services on all)

- Snap, power back up.

This makes sense, because it allows for a clean recovery point across the domain, avoiding the obvious issues you'll run into re: PSC replication. But, it's pretty inconvenient. If you have a failed upgrade on one vcenter, be prepared to roll them all back.

The potential scenario I'm looking at is a 9 x VCSA, single SSO deployment. (3 x tenants, 3 datacenters). To me, this spells bad news. Yes, I want centralised auth, I want global object searching....but I don't think I have enough confidence in VMware's directory service, nor do I think there's enough expertise out there to support this appropriately.

Interested to know if anyone here has a large linked mode environment and how this impacts routine patching an upgrades? It's crazy right? Someone convince me otherwise!

Re: Configure SAML auth / SSO with Skyline Advisor?

mrstorey303 — Wed, 08 Jul 2020 09:32:43 GMT

Thanks - unfortunately there's no appetite to deploy another IDP - we're heavily invested in Okta so we'd like to use this really.

It looks entirely possible to do - I just could use some advice with what values to set for the login redirect URIs on both sides / the SP and the IDP.

I have a ticket open with VMware support but I think this is uncharted territory for them - they're asking re: 'What errors are you getting' etc. I'm not really at that point - I could guess some values to generate errors, but not sure if that's a sensible way to progress the conversation!

I'll keep hacking away, but if there's any bright spark out there who's done this, I'd appreciate talking to them!

Re: Scheduled Findings / Recommendations Reports

mrstorey303 — Thu, 25 Jun 2020 07:33:00 GMT

Excellent, thank you.

Re: Configure SAML auth / SSO with Skyline Advisor?

mrstorey303 — Tue, 23 Jun 2020 10:11:45 GMT

Perfect! many thanks - forgive my ignorance, but these guides are specifically for Velocloud / VMware's SD-WAN products - presumably they're provided here because both Velocloud and Skyline are services which run in the same cloud service, and are therefore subject to the same authentication options?

Thanks

Re: UAG Blast Tunnelling for HTML Connections Only?

mrstorey303 — Mon, 22 Jun 2020 08:44:06 GMT

OK thanks, this is what I figured.

In which case, I guess my options are for internal TrueSSO are:

- Enable tunnelling - both thick clients and HTML5 clients are tunnelled regardless.

- Disable tunnelling, but configure the connection servers to return a DNS name rather than IP address, so that direct HTML5 connections do not throw a cert warning. (Not entirely happy to do this, because it'd introduce a significant dependancy on accurate DNS queries for instant clones / highly ephemeral environments. I could see an issue where users are brokered out to different machines to what the connection server allocated!)

I should probably throw in a feature request for the UAG product team. Just as a side, I've always wondered why these cert warnings only throw for brokered, direct HTML5 connections, and not brokered direct connections from the thick client? Maybe a question for another thread!

Thanks

Scheduled Findings / Recommendations Reports

mrstorey303 — Sat, 20 Jun 2020 05:04:45 GMT

Is it possible to configure Skyline advisor to set emails out on a schedule? ie - Send me a weekly schedule of all critical + moderate severity findings in my environment to x email addresses?

The only config option I see is a simple toggle switch 'Email New Critical Findings', which is great - I'd want this to continue, but I see some value in having regular reports, almost to nag our engineers into resolving or acknowledging existing findings / recommendations.

Likewise, are there any plans to offer an API to query Skyline via internal automation? Feel there could be come value in exposing / feeding Skyline data to internal monitoring tools.

I'm trying to develop a process for our engineers to regularly review Skyline - scheduled emails and API access would help here I think.

Thanks