MKguy Tracker

Re: Using HA, how to get VM to go back to original host?

MKguy — Tue, 01 Nov 2016 12:11:01 GMT

Usually people use DRS and VM-to-Host affinity rules to migrate the VM back to a preferred host. You need DRS which is only included in enterprise plus licenses.

See the DRS documentation:

Using DRS Affinity Rules

If you don't have enterprise plus licenses for DRS then you can script something with basic PowerCLI.

Re: ESX host to vCenter network traffic

MKguy — Fri, 28 Oct 2016 13:06:10 GMT

How much bandwidth are we talking about exactly here?

There shouldn't be too much traffic between vCenter and ESXi hosts, it's mostly management traffic and ESXi sending periodic performance data of host and VM objects to vCenter. Depending on how many VMs run on that host and what your vCenter statistics collection level is, the amount of performance data being sent data can vary a lot:

Especially statistics level 3 and 4 will generate a lot of data and also bloat your vCenter database: Data Collection Levels

If you have VUM on vCenter as well then pushing patches can take up a lot of bandwidth, but I don't think you're doing that.

Re: NIC Speed Issue of VMNIC7 - Cable is Cat6

MKguy — Thu, 27 Oct 2016 20:25:57 GMT

I'm pretty sure the cable type is not the problem here. It might be a faulty cable (unlikely) but otherwise it should work just as well as cat5e or cat7.

Check the physical switch port configuration where your vmnic7 is attached to. Make sure both ends are configured for auto-negotiation of speed AND duplex. A mismatch will typically cause a fallback to a lower speed.

You can confirm this pretty quickly if you just swap the cables on the host-end between vmnic7 and another NIC that is running at 1000/full. If the other NIC gets only get 100 now then the switch port probably has misconfigured speed/duplex settings.

Re: SMTP settings

MKguy — Tue, 25 Oct 2016 10:08:38 GMT

Unfortunately vCenter supports neither SMTP authentication nor SSL/STARTTLS at this point. This is confirmed by VMware in the following KB article:

Support for SMTP authentication and STARTTLS in the vCenter Server and vCenter Server Appliance 5.x and 6.0 (2063147) | …

As a workaround you could setup a simple SMTP forwarding server that takes the unauthenticated and unencrypted mails from vCenter and forwards them with authentication and encryption to you production mail server. If you run vCenter on Windows you could just install the builtin Windows SMTP service.

Re: VMDK disk as Windows Cluster

MKguy — Thu, 20 Oct 2016 22:36:14 GMT

Yes, when running clusters across hosts with RDMs you have to use an additional SCSI controller for the cluster disks and set its SCSI bus sharing mode to physical.

The detailed steps are explained in the MSCS guide starting from page 16 on the "Cluster Virtual Machines Across Physical Hosts" chapter and in the checklist on page 27:

http://pubs.vmware.com/vsphere-60/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-60-setup-mscs.pdf

Re: VMDK disk as Windows Cluster

MKguy — Wed, 19 Oct 2016 13:48:04 GMT

Yes, RDM implies you use the host-side passthrough of a physical block storage device presented to the ESXi host as opposed to in-guest iSCSI, which is transparent to the host.

Both types support the cluster-across-boxes configuration.

Re: VMDK disk as Windows Cluster

MKguy — Wed, 19 Oct 2016 12:09:02 GMT

Yes, you can build Windows Failover Clusters with shared VMDK disks sitting on normal VMFS volumes. Shared VMDKs on NFS however are not supported.

The caveat is that the shared VMDK on VMFS option is only formally supported for "Cluster-in-a-box" configurations where all VMs must reside on the same ESXi host.

This is probably not what you want for a clustered service where high availability is key. For "Cluster-acorss-boxes" with shared disks you have to use RDMs or in-guest ISCSI/SMB.

See these documents for more details:

Microsoft Clustering on VMware vSphere: Guidelines for supported configurations (1037959) | VMware KB

Microsoft Cluster Service (MSCS) support on ESXi/ESX (1004617) | VMware KB

http://pubs.vmware.com/vsphere-60/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-60-setup-mscs.pdf

Re: The VIBs was Skipped when upgrade scsi_megaraid_sas from 6.505 to 6.506

MKguy — Tue, 18 Oct 2016 10:04:30 GMT

First off, you should use the update instead of install command when updating existing VIBs, though this should be irrelevant for your issue just as a general recommendation.

Looking at just the build number it seems to be identical, both have 472560.

Here's a newer version of the driver version 6.610.15.00-5.5-3342447:

https://my.vmware.com/group/vmware/details?downloadGroup=DT-ESXI55-LSI-SCSI-MEGARAID-SAS-66101500-1OEM&productId=353

Download the offline package instead of the VIB file and run the following commands and post the output here:

# esxcli storage core adapter list

# esxcli software sources vib list -d /tmp/megaraid_sas-6.610.15.00-5.5-offline_bundle-3342447.zip

# esxcli software sources vib get -d /tmp/megaraid_sas-6.610.15.00-5.5-offline_bundle-3342447.zip

This will tell you a lot of information from the metadata contained in the offline bundle. It will also tell you whether this is considered an upgrade, a downgrade or if the same version is already installed.

To actually upgrade then run:

# esxcli software vib update -d /tmp/megaraid_sas-6.610.15.00-5.5-offline_bundle-3342447.zip

Re: Block ssh and ICMP for VMs on VDS

MKguy — Mon, 17 Oct 2016 19:54:29 GMT

Yes, rules defined on individual dvports only apply to that dvport and not to the entire port group.

Just be aware that with filtering both Ingress/Egress, your VM will not be able to establish outboud SSH connections either. If you just want to filter SSH connections TO and not FROM the port you need to set the rule to Egress only as Egress/Ingress traffic is always seen from the VDS point of view.

Re: Question regarding NPIV

MKguy — Mon, 17 Oct 2016 10:16:06 GMT

This is a common misunderstanding with NPIV:

NPIV does NOT present a virtual FC HBA to your VMs. Your VM will only see an ordinary disk on an ordinary SCSI controller you selected. It's a logical abstraction allowing you to map virtual WWNs to VMs, but this abstraction is done on the host-side transparently for VMs.

Please see these similar threads:

NPIV question

Presenting Fiber HBA's to VM under ESXi 6

Re: Block ssh and ICMP for VMs on VDS

MKguy — Mon, 17 Oct 2016 10:02:42 GMT

You have to select the appropriate layer 4 transport protocol, which is TCP in this case. Then you can enter the layer 4 destination port to match, which is port 22 for SSH:

Re: Block ssh and ICMP for VMs on VDS

MKguy — Fri, 14 Oct 2016 18:51:41 GMT

If you're on 5.5 or later you can use the traffic filtering feature of the VDS, which allows you to define basic ACLs based on Layer 2 and 3 Source/Destination as well as layer 4 ports. Note that you need to use the Web Client to access these settings in the VDS.

Refer to the documentation for more information on how to enable traffic filtering:

Filter Traffic on a Distributed Port Group or Uplink Port Group

I would prefer implementing a proper firewall though.

Re: What is a express patch ?

MKguy — Thu, 13 Oct 2016 17:15:25 GMT

I agree with you that the ESXi patch terminology and versioning are very confusing.

What you need to be aware of is that ESXi patches are cumulative - they always update the entire esx-base VIB package. This means Patch 10 includes not only the fixes it was developed for, but also all fixes from patch 1-9. This includes any earlier "express" patches. There is no need to apply earlier esx-base patches prior to updating to the latest package because all fixes are included in the full cumulative package anyways.

I would just get the latest Cisco offline bundle which includes other packages like drivers and update to this version:

https://my.vmware.com/web/vmware/details?downloadGroup=OEM-ESXI55U3B-CISCO&productId=353

And then update to the latest ESXi patches. You can apply any normal VMware ESXi patches on top of specific ISO installations because 3rd party vendors like Cisco, HP, Dell etc do not modify any actual ESXi code. They just add management extensions and update drivers to specificially support their hardware. Anyone can create identical ISOs by grabbing all the individual packages from VMware and the vendors and combine them together in one ISO/offline bundle.

The ESXi build number (vmware -vl) corresponds to the esx-base VIB version (esxcli software vib list | grep base). You can check which current patch level you have by matching the build number against this list:

Build numbers and versions of VMware ESXi/ESX (2143832) | VMware KB

Re: ESXCLI shows Receive Packets Dropped on vmk0. Is this normal?

MKguy — Wed, 12 Oct 2016 18:38:38 GMT

Try this recently released patch which fixes a false drop reporting issue. While it mentions performance charts, it might help with your issue as well:

VMware ESXi 6.0, Patch ESXi-6.0.0-20160804001-standard (2145667) | VMware KB‌

A newer ixgbe driver version 4.4.1 is available as well:

https://my.vmware.com/group/vmware/details?downloadGroup=DT-ESXI60-INTEL-IXGBE-441&productId=491

You should check for firmware updates while you're at it too.

Doing some math with the port stat numbers shows you have 347,884,869 (Unicast) + 1,214,237 (Broadcasts) = 349,099,106 RX packets total

Meanwhile you have 430,499 dropped RX packets, which equals a drop rate of 0.12%.

My bet is still rather on a reporting issue or filtering of irrelevant broadcast/multicast traffic, but even if it's real, then this should't cause noticeable issues, especially not on VMs which are separate from a vmk interface.

Re: ESXCLI shows Receive Packets Dropped on vmk0. Is this normal?

MKguy — Wed, 12 Oct 2016 17:23:20 GMT

Where are these drops reported? These may be false positives:

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2052917

Do you see any DRPRX in the esxtop network view?

What ESXi version, your physical NIC type, firmware and drivers? Please provide the output of the following commands on the ESXi shell to give us more information:

# vmware -vl

# esxcli network nic list

# esxcli network nic get -n vmnicX

Check the virtual port packet counters of your vmk0 in vsish as described here and post the output as well:

https://www.reddit.com/r/vmware/comments/3l306b/esxi_6_and_receive_packets_dropped_counter/

Even if there were drops on your vmk0, this should technically have no impact on the separated VM network traffic.

Re: The most basic VSAN question

MKguy — Wed, 12 Oct 2016 16:09:36 GMT

do I have to RAID my physical drives before I can use them for vsan?

No, it's actually more recommended to use JBOD/passthrough mode on the storage controller and only configure RAID0 if you have to.

1x ssd for ESXi (on host)

Running the ESXi system from a SSD is a real waste of money and resources. A USB SD-card is enough as a boot drive for an ESXi host, as it runs from a RAMDISK after booting anyways.

According to the HCL you need the aacraid driver for your 71605Q controller. What you linked and described is for management/monitoring CIM provides and not the actual drivers.

Try installing this recent aacraid-6.0.6.2.1.50667 driver:

https://my.vmware.com/web/vmware/details?downloadGroup=DT-ESXI60-PMC-AACRAID-62150667&productId=491

After installing you can confirm the version with

# esxcli software vib list | grep aacraid

If that doesn't make the drives work in JBOD/passthrough mode then you may have to use a RAID0 on top or switch to a different controller.

Re: How do I log what hardware setting were changed on VM from what to what (vCPU, RAM, harddisks a.o.)?

MKguy — Wed, 05 Oct 2016 09:09:50 GMT

Unfortunately the data from vCenter tasks and events is very generic and hard to make sense of in detail or for audit purposes.

At VMworld they demoed a tech preview for the next major release with logging enhancements that are supposed to tackle these shortcomings:

INF8845 - vSphere Logs Grow Up! Tech Preview of Actionable Logging with vRealize Log Insight

Re: vmware encryption

MKguy — Thu, 29 Sep 2016 13:38:43 GMT

vSphere 6 currently does not offer any form of storage or VM-level encryption features. This might change with a future release (try to get in the beta program), but at the moment you only have two options:

1. Encryption at the physical array/disk level

2. Encryption inside the Guest OS

Re: Command line tool for datastore performance test in ESXi 6

MKguy — Thu, 29 Sep 2016 12:57:09 GMT

As you noted there is no such thing and it's not easy to port an existing one over to ESXi, since it's Linux.

But even if there was, don't run benchmarks inside the limited ESXi base OS. You would probably get some very wonky numbers.

As a type-1 Hypervisor, the ESXi userspace shell based on a busybox environment is just a limited interface for management purposes separate from the actual vmkernel. It's limited in terms of resources and IO scheduling.

I know this might seem counter-intuitive at first, but benchmarking from a VM will actually allow you to make use of the full host's capabilities, since that IO really runs in vmkernel space from the host-perspective.

If you still really insist you want to test your hardware from a baremetal point of view, then boot the server into a live Linux of your choice and test there.

Re: cisco discovery protocol

MKguy — Thu, 08 Sep 2016 21:39:55 GMT

I agree that using a VM with Wireshark or tcpdump would be far better, but unfortunately this probably won't work in this case, even with the catch-all 4095 VLAN and promiscuous mode.

Just like its LLDP counterpart, CDP is link-layer protocol using a special multicast destination MAC - frames will not be forwarded between directly attached links, i.e. they will be terminated at the physical NIC. This is the reason why you need to use the pktcap-uw tool instead of tcpdump-uw (which only operates on vmkernel NICs) on the shell.

If the CDP multicast frames were forwarded like normal frames on the network, you would see information about every switch in the same layer 2 domain on every port at the same time.