I recently ran into an issue upgrading a customer's vSphere environment. This environment had 4 older servers running ESX 4.0 in a single cluster that were to be replaced with new hardware running ESXi 4.1. The upgrade process was basically to enable EVC on an existing cluster, remove one host at a time and finally add one host at a time back to the cluster. When it was time to add the first host back into the existing cluster, I received the error "Cannot complete the configuration of the HA agent on the host. Misconfiguration in the host network setup." A quick search of the VMware Knowledge Base turned up kb 1019200 - "Configuring VMware High Availability fails with the error: Cannot complete the configuration of the HA agent on the host"
The kb article states that "This issue occurs if all the hosts in the cluster do not share the same service console or management network configurations. Some hosts may have service consoles using a different name or may have more service consoles than other hosts." This makes perfect sense, as the ESX 4.0 hosts are using a Service Console connection type for the management network and the new ESXi 4.1 host is using a VMkernel connection type. Fortunately, the kb article went on to discuss how to use the "das.bypassNetCompatCheck" option for HA to ignore this mismatched management connection condition. Basically, turning off HA for the cluster and then turning it on again with the "das.bypassNetCompatCheck" option allowed the ESXi 4.1 host to join the existing cluster without incident. Once the fourth and final host was added to the cluster and the cluster contained only ESXi 4.1 hosts, the "das.bypassNetCompatCheck" was removed using the same procedure.
Hopefully this information will help others, as I am certain this upgrade scenario is a common one.
Thanks for reading,