virtual_dave1 Tracker

Re: VCSA 6.7U1 task name suffix .label

virtual_dave1 — Sat, 24 Nov 2018 20:11:00 GMT

Scrap that - I had cause to reboot it again and bizarrely for whatever reason the task are now displaying properly.

I would add that before posting I did reboot the affected VCSA at least once. But I did not reboot both VCSAs and both external PSCs - so perhaps that is somehow relevant.

Anyway - thanks. Have some points

Dave

Re: VCSA 6.7U1 task name suffix .label

virtual_dave1 — Sat, 24 Nov 2018 17:47:24 GMT

Thanks for the suggestion but sadly it made no difference. Bounced both PSCs and both VCSAs and the issue persists.

VCSA 6.7U1 task name suffix .label

virtual_dave1 — Sat, 24 Nov 2018 16:17:30 GMT

Lots of hits for this on Google for 6.5 but not for 6.7.

Just done a seemingly successful upgrade of a two-site environment vcsa\psc 6.5U2 to 6.7U1. So far so good.

However one of the vCenters is showing task names with the suffix '.label', for example, vm.Snapshot.remove.label, VirtualMachine.relocate.label. Interestingly if I connect to the other vCenter (in ELM, different site and different PSC) I see the same tasks displaying normally.

As per a couple of the Google hits, I've checked the MOB of the affected vCenter and there are no third-party extensions installed.

Anybody else seen this with 6.7U1, or have any suggestions?

Thanks.

Dave

Re: vSphere 6.0U3 EAM Error

virtual_dave1 — Thu, 07 Sep 2017 08:18:42 GMT

Thanks for taking the time to post Rajeev.

But i already went through that article before posting my question. None of the symptoms listed seem to apply to my problem shown in the logs above.

In a nutshell, my vmware-eam service on VCSA will not start.

vSphere 6.0U3 EAM Error

virtual_dave1 — Wed, 06 Sep 2017 15:27:48 GMT

Hi all

I have a problem with EAM. We have been using NSX guest introspection and it started playing up recently, which attracted our attention.

In vSphere Web Client, under Administration\vSphere ESX Agent Manager \ Solution I get the following error:

503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http16LocalServiceSpecE:0x7fdc140fea00] _serverNamespace = /eam _isRedirect = false _port = 15005)

On VCSA, the VMware-eam service is not running. Starting it results in an error, the details of which are found in /var/log/VMware-eam/wrapper.log

STATUS | wrapper | 2017/09/06 15:45:55 | --> Wrapper Started as Daemon

STATUS | wrapper | 2017/09/06 15:45:55 | Java Service Wrapper Professional Edition 64-bit 3.5.24

STATUS | wrapper | 2017/09/06 15:45:55 | http://wrapper.tanukisoftware.com

STATUS | wrapper | 2017/09/06 15:45:55 | Licensed to VMware Global, Inc. for VMware vCenter ESX Agent Manager

STATUS | wrapper | 2017/09/06 15:45:55 |

INFO | wrapper | 2017/09/06 15:45:55 | Event Command 'wrapper_start': Command line[0] : /usr/sbin/cloudvm-ram-size

INFO | wrapper | 2017/09/06 15:45:55 | Event Command 'wrapper_start': Command line[1] : -J

INFO | wrapper | 2017/09/06 15:45:55 | Event Command 'wrapper_start': Command line[2] : vmware-eam

INFO | wrapper | 2017/09/06 15:45:55 | Event Command 'wrapper_start': Command line[3] : -O

INFO | wrapper | 2017/09/06 15:45:55 | Event Command 'wrapper_start': Command line[4] : /etc/vmware/java/vmware-eam_jvm.conf

INFO | wrapper | 2017/09/06 15:45:55 | Event Command 'wrapper_start': Command launched (pid: 23310), blocking for up to 3600 seconds...

INFO | wrapper | 2017/09/06 15:45:56 | Event Command 'wrapper_start': Command completed with exit code: 0

INFO | wrapper | 2017/09/06 15:45:56 | Event Command 'wrapper_start': Command completed with exit code: 0 Continuing.

INFO | wrapper | 2017/09/06 15:45:56 | Event Command 'jvm_prelaunch': Command line[0] : /opt/vmware/bin/python

INFO | wrapper | 2017/09/06 15:45:56 | Event Command 'jvm_prelaunch': Command line[1] : ../wrapper/bin/update_pnid.py

INFO | wrapper | 2017/09/06 15:45:56 | Event Command 'jvm_prelaunch': Command line[2] :

INFO | wrapper | 2017/09/06 15:45:56 | Event Command 'jvm_prelaunch': Command line[3] :

INFO | wrapper | 2017/09/06 15:45:56 | Event Command 'jvm_prelaunch': Command line[4] :

INFO | wrapper | 2017/09/06 15:45:56 | Event Command 'jvm_prelaunch': Command launched (pid: 23313), blocking for up to 15 seconds...

INFO | wrapper | 2017/09/06 15:45:56 | Event Command 'jvm_prelaunch': Command completed with exit code: 1

INFO | wrapper | 2017/09/06 15:45:56 | Event Command 'jvm_prelaunch': Command completed with exit code: 1 Continuing.

STATUS | wrapper | 2017/09/06 15:45:56 | Launching a JVM...

INFO | jvm 1 | 2017/09/06 15:45:56 | Picked up JAVA_TOOL_OPTIONS: -Xms32M -Xmx128M

INFO | jvm 1 | 2017/09/06 15:45:56 | WrapperManager: Initializing...

INFO | jvm 1 | 2017/09/06 15:45:57 | WrapperSimpleApp:

INFO | jvm 1 | 2017/09/06 15:45:57 | WrapperSimpleApp Error: Encountered an error running main:

INFO | jvm 1 | 2017/09/06 15:45:57 | WrapperSimpleApp Error: org.springframework.beans.factory.BeanDefinitionStoreException: Invalid bean definition with name 'esxAgentMgr' defined in class path re source [eam-server-beans.xml]: Could not resolve placeholder 'eam.hostd_restart_timeout'

INFO | jvm 1 | 2017/09/06 15:45:57 | WrapperSimpleApp Error: at org.springframework.beans.factory.config.PropertyPlaceholderConfigurer.processProperties(PropertyPlaceholderConfigurer.java:268 )

INFO | jvm 1 | 2017/09/06 15:45:57 | WrapperSimpleApp Error: at org.springframework.beans.factory.config.PropertyResourceConfigurer.postProcessBeanFactory(PropertyResourceConfigurer.java:75)

INFO | jvm 1 | 2017/09/06 15:45:57 | WrapperSimpleApp Error: at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java: 554)

INFO | jvm 1 | 2017/09/06 15:45:57 | WrapperSimpleApp Error: at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java: 528)

INFO | jvm 1 | 2017/09/06 15:45:57 | WrapperSimpleApp Error: at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:363)

INFO | jvm 1 | 2017/09/06 15:45:57 | WrapperSimpleApp Error: at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)

INFO | jvm 1 | 2017/09/06 15:45:57 | WrapperSimpleApp Error: at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:83)

INFO | jvm 1 | 2017/09/06 15:45:57 | WrapperSimpleApp Error: at com.vmware.eam.util.Main.main(Main.java:50)

INFO | jvm 1 | 2017/09/06 15:45:57 | WrapperSimpleApp Error: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

INFO | jvm 1 | 2017/09/06 15:45:57 | WrapperSimpleApp Error: at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

INFO | jvm 1 | 2017/09/06 15:45:57 | WrapperSimpleApp Error: at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

INFO | jvm 1 | 2017/09/06 15:45:57 | WrapperSimpleApp Error: at java.lang.reflect.Method.invoke(Unknown Source)

INFO | jvm 1 | 2017/09/06 15:45:57 | WrapperSimpleApp Error: at org.tanukisoftware.wrapper.WrapperSimpleApp.run(WrapperSimpleApp.java:290)

INFO | jvm 1 | 2017/09/06 15:45:57 | WrapperSimpleApp Error: at java.lang.Thread.run(Unknown Source)

STATUS | wrapper | 2017/09/06 15:45:59 | <-- Wrapper Stopped

So following my nose, a potentially relevant snip from the aforementioned eam-server-beans.xml found in /usr/lib/VMware-eam/conf is:

<bean id="esxAgentMgr"

class="com.vmware.eam.EsxAgentManagerImpl"

lazy-init="false"

depends-on="springContext,

vcInventory,

vcListener,

httpNfcLeaseTimeoutHandler,

nameManager,

vcDatastoreManager,

vcEventManager,

vcKeyValueStore,

l10n">

<constructor-arg ref="adapterServer" />

<constructor-arg value="${vc.proxy.host}" />

<constructor-arg value="${vc.proxy.port}" />

<constructor-arg value="${vc.tunnelSdkUri}" />

<constructor-arg ref="clientAuthenticator" />

<constructor-arg value="${eam.clear_db_on_startup}" />

<constructor-arg value="${eam.support_linked_clone}" />

<constructor-arg value="${eam.debug_ref_count}" />

<constructor-arg value="${vum.integration}" />

<constructor-arg value="${Net.DVFilterBindIpAddress}" />

<constructor-arg value="${Net.TrafficFilterIpAddress}" />

<constructor-arg value="${Net.DVFilterVswifIpAddress}" />

<constructor-arg ref="commonMapStore" />

<constructor-arg ref="vpxdConnection" />

<constructor-arg ref="vumConnection" />

<constructor-arg ref="nameManager" />

<constructor-arg ref="vibPackageDepots" />

<property name="hostdRestartTimeout" value="${eam.hostd_restart_timeout}" />

</bean>

I'm struggling as I need to get my NSX piece back up and running (ironically so I can gracefully remove it), but believe I need the EAM to be working properly first.

Any ideas before I open a ticket?

Thanks.

vDave

esxcli - agent vm settings

virtual_dave1 — Fri, 06 Jan 2017 08:56:29 GMT

Is it possible to configure ESXi's "Agent VM Settings" values for datastore and/or network via esxcli (or some other command line method)?

Looking to incorporate it into a build script so NSX will auto-deploy agent VMs to new hosts.

Thanks.

Re: NUMA question

virtual_dave1 — Thu, 15 Oct 2015 13:27:33 GMT

Thanks MKGuy - very interesting. I will digest and look at the links you have sent.

NUMA question

virtual_dave1 — Thu, 15 Oct 2015 10:41:44 GMT

Hope this is the right place for this. Apologies up front if i'm using the wrong lingo, misunderstanding concepts etc. Please correct me.

Question about NUMA and performance.

I have BL460 gen9 with 2x 10-core Xeons and currently 160GB RAM.

However - the workload we now want to run on these hosts consists of 8-core VMs running Citrix Xenapp 7.6 delivered via PVS (using RAM write-cache). We desire 32GB RAM per VM. Physical RAM can be increased. The VM sizing is the result of testing and recommendations from Citrix.

So as it stands the NUMA nodes are:

1x 10-core Xeon (20 logical procs)
80GB RAM

We want the VMs to be:

8x vCPU
32GB RAM (including 8GB non-paged-pool PVS write cache)

Clearly these servers weren't bought for this specific purpose - 8 doesn't go into 10 (or 20) very well, and 32 doesn't go into 80 very well either! But we have what we have.

So lets say i spin up 4 VMs on this host. Presumably they will utilise 32 of the 40 logical procs (8vCPU x 4VMs) and 128GB (32GB x 4VMs) of the 160GB of RAM. I'm guessing ESXi will spread these across both NUMA nodes, so there will be 2x VMs on each socket, which leaves 4 logical processors and 16GB pRAM 'unused' on each of the two NUMA nodes.

If i spin up a 5th VM of the same spec, there is enough physical resource to satsify it however both CPU and RAM will be split equally between the two NUMA nodes right? 4xCPU and 16GB from each to make up the 8xvCPU and 32GB VM.

Presumably i am going to see some performance degredation on this VM that spans?

Is this still measurable and noticeable on a modern host such as this?

NUMA is about memory right? So if i make sure each socket has enough RAM to satisfy 3 (instead of 2) VMs worth of workload on it, then potentially I could have 3 VMs localised on each NUMA node, with a modicum of over-comittment on the CPU (24vCPUs running on 20 logical CPUs)? Our Citrix workload is far more RAM-heavy than CPU-heavy.

Any thoughts or advice?

Thanks.

Dave

PSC 6 - remove old PSC

virtual_dave1 — Thu, 17 Sep 2015 14:26:54 GMT

Some time ago I attempted to install the first external PSC on Windows into my existing multi-site 5.5 SSO. Despite effort i could not get it to complete successfully so gave up, used a PSC appliance, moved the rest of the environment to 6.0 and all has been well since. I forget what the exact problem was.

Zoom forward to today, all is working. I have 2x external PSC appliances and 2x VCSA appliances.

I attempted to upgrade to Update 1 but despite successful upgrades of PSCs and VCSAs I could not get the web client to load up after login (legacy client worked fine). "An internal error has occurred - index 0 out of bounds'. Web-client log files kept showing errors about not being able to contact this stale\old Windows PSC. I was under pressure to get access to vCenter back so i rolled back to snapshots i took before starting upgrade and all is well with the world again.

Before attempting to upgrade again I would like to clear up this stale record but am having trouble. I've been through KB2106736 but i get the following error:

cmsso-util unregister --node-pnid Old-PSC --username administrator@vsphere.local --passwd sso_pwd

Could not find a host id which maps to Old-PSC in Component Manager

Failed!!!

I also try the following but get the below error:

/usr/lib/vmware-vmdir/bin/vdcleavefed -h Old-PSC -u administrator -u sso_pwd

fdcleavefd offline for server Old-PSC

Leave federation cleanup failed. Error[13] - Confidentiality required.

I have run the above commands on both my PSC appliances with the same result.

Any ideas before I open a support case?

Thanks.

Dave

VCSA 6 - /dev/mapper/seat_vgseat filling up

virtual_dave1 — Tue, 08 Sep 2015 22:50:18 GMT

Fresh install of vcsa 6 into an existing multi-site SSO from 5.5u2 (now SSO 6).

The seat_vgseat volume keeps filling up. It has filled 20GB in roughly 36 hours. Other volumes seem fine (<50% usage).

Any ideas?

My environment consists of 2 sites, 2x vcsas and 2x psc appliances. There is also a Windows vCenter 6 with embedded PSC (upgraded from 5.5u2) in the same site as the offending vcsa while i finish migrating. VCSA in the other site is not exhibiting this problem (vg_seat vol is c. 200MB usage).

I'm guessing there is an influx of events coming into the VCSA and filling the volume?

If so - can I get some visibility into what might be generating the events? (or stats or tasks?)

Any advice?

Thanks.

Re: vShield Endpoint Thin Agent

virtual_dave1 — Mon, 17 Feb 2014 23:09:15 GMT

Are the 4 VMs that do appear in SEP all running on the same host?

VCNS 5.5 - Endpoint 5.1?

virtual_dave1 — Sun, 16 Feb 2014 18:39:05 GMT

Was running vSphere 5.1 with VCNS 5.1 running Endpoint. No problems.

Upgraded vCenter etc. to 5.5 a while back, the first step of which (according to KB2057795) was to upgrade vShield manager. Hosts (and thus Endpoint integration) still on 5.1. No problems.

I've just moved my hosts to 5.5 this weekend (fresh installs, not upgrades), and have gone to install the Endpoint integration from vShield Manager, before then upgrading Tools etc. However, vShield Manager (5.5.0a build 1473628) appears to only want to install vShield Endpoint 5.1.0-01255202. It does, however, offer vShield App 5.5.0-1447281.

Is vShield Endpoint 5.1.0-01255202 the latest version, and the correct version for a vShield Manager 5.5.0a to be offering to 5.5 hosts? Or have I done something wrong?

Thanks.

dave

Re: ESXi 5.1 Scratch vs VM Swap Locations

virtual_dave1 — Thu, 20 Jun 2013 09:16:29 GMT

OK thanks. So at this current moment in time, would it be fair to say it is not possible to set the VM Swapfile Location property (in any useful way) via command line? (assuinng vsphere 5.1U1 and vcenter is in use).

We configure vswitches etc. via esxcli during build, and when the host is added to vcenter the information is synchronised from the host to VC. It is a shame this setting can't also be sync'd from host to VC. Or am I missing something?

Perhaps this can be done by a vCO workflow after the host is added to vcenter. I will explore. Thanks for your time.

Re: ESXi 5.1 Scratch vs VM Swap Locations

virtual_dave1 — Thu, 20 Jun 2013 08:53:58 GMT

Thanks AakashJ. That works as you describe.

My goal is (was) to use it in a kix build script but seemingly that will be difficult.

What is the reason for it being reset when the host is added to vCenter? Just for my understanding.

ESXi 5.1 Scratch vs VM Swap Locations

virtual_dave1 — Wed, 19 Jun 2013 20:25:54 GMT

I'm running ESXi5.1U1 from an SD card with no local persistent storage.

I configure persistent scratch location as per KB1033696. No problem var,log,download,core folders created and all is well.

Now i want to configure VM Swap file location, through the GUI i would go to "Configuration\Software\Virtual Machine Swapfile Location" and point it to a VMFS datastore. No problem.

My question is - how do i set the VM Swapfile location via command line?

Thanks.

vShield 5.1 Endpoint VMCI Driver for Linux?

virtual_dave1 — Wed, 03 Oct 2012 08:52:33 GMT

Can't find anything that explicitly states support, or lack thereof, for Linux guests.

Is there a vShield VMCI driver (Endpoint Thin Agent) for Linux yet? Our flavours are Debian and CentOS.

If not, is it on the roadmap? (if not, why not out of interest?)

Thanks.

Re: ESX5.1 immature: Converter and VMwareTools problems

virtual_dave1 — Tue, 25 Sep 2012 10:35:02 GMT

Awesome - i am also getting

[ warning] [vmusr:vmtoolsd] Failed registration of app type 2 (Signals) from plugin unity.

on other VMs. First one i found was a WSUS VM if that is in any way relevant.

I have SR open re: RPC Loop error, i have provided logs etc, will update thread if i hear anything of significance.

My VMTools version is 9.0.0 build 782409.

Re: ESX5.1 immature: Converter and VMwareTools problems

virtual_dave1 — Mon, 24 Sep 2012 11:51:58 GMT

I too am getting [ warning] [vmusr:vmusr] Error in the RPC receive loop: RpcIn: Unable to send.

On our XenApp VMs no less - useful!!!

Opening a ticket now.

Linux Guest network settings customisation

virtual_dave1 — Thu, 13 Jan 2011 15:09:26 GMT

Hi all

I have SRM 4.1 running and customising Windows guests' networking great. Having problems with Linux customisation though and wondered if anybody could point me in the right direction.

It's all Debian 5 x64, about 15 VMs.

I've done a lot of searching and am aware that VSphere 4.1 is allegedly the last to support customisation of Debian 5 guests. I have tried in vain to get a custom spec applied to a Linux VM, either by SRM or by manually cloning etc. Clone and/or vm recovery works fine but no customisation appears to be performed on the Linux VMs. So given the lack of future support for custom specs, perhaps the answer lies in some scripting somewhere. I am no scripting guru but routinely knock up vbscripts etc. for the windows environment so i am perhaps familiar with concepts.

How do you guys customise your Linux guests' networking settings during SRM failover or testing? If somebody could explain the 'mechanics' of how they do it in suitably basic terms for my small brain that would be greatly appreciated! And any examples would be amazing!

eg. I'm guessing that probably the least involved method would be to have a script on each VM that is then remotely executed by SRM in a call-out script. The guest-based script could even just copy pre-configured interfaces and resolv.conf files. In this scenario what actual process would run the script inside the guest? Does VCenter speak to the guest VMware Tools, and then vmware tools executes the script? In high-level terms, how does an SRM callout script running on Windows Server ultimately execute a command on a Linux VM?

Not after being spoon-fed but I'm fairly new to this and my knowledge of Linux is verging on embarrassing at present and have drawn blanks with google and the vmware communities. Aware of Perl, PowerCLI, even VIX options but looking for a leg-up if poss.

Thanks.

Re: SRM 4.1 client problem

virtual_dave1 — Tue, 23 Nov 2010 12:31:28 GMT

No no joy yet, had a call from vmware support but couldn't immediately get to the bottom of it, have sent some more info to them (netstat results) but not heard back yet. Am chasing up as my storage is replicated and i'm ready to roll!

Tried pairing the sites from the working DR SRM installation but it cannot connect to the primary site and errors out saying "could not connect to :80".

Will make sure to update if i get anywhere.

Good luck.