VMware Cloud Community
IvarHome
Hot Shot
Hot Shot

What means requirement "vCenter and ESXi must reside in the same subnet"?

Why? Do they communicate with L2 level, without IP? Otherwise I dont see any reason why they must reside in the same subnet. Im tired of all this VMWare uncompleted declarations.

12 Replies
Shawnlo
Contributor
Contributor

I am interested in seeing where it says this, can you point to the doc?

In my labs I have no issues running ESXi in separate subnets connected to vCenter the only requirement would be routing.

My thought is it would be for simplification due to possible routing issues.

IvarHome
Hot Shot
Hot Shot

This is good news. I dont remenber where, I readed it many times. Whole VMware documentation is one piece of shit, its absolutely useless. I just experiment one special job. I have ESXi-s joined to vCenter by IP-s. So, I have no any chance to connect vCenter with ESXi through different vmkernel with different IP. In other side IP is of course faster method than FQDN and works even when DNS is down. But one possibility still exist, altough its very uncmfortable. I can set up two firewalls, between connected through new subnet and both firewalls uplinks in the same subnet as it is now between ESXi and vCenter. Both firewalls make SNAT and DNAT. So, ESXi and vCenter sees each other as in the same subnet, both firewalls uplinks looks like they are ESXi and vCenter IP-s, but I can configuse DNAT to connect also with any IP in ESXi. Its hard to set up. One problem is to access firewalls for configuration and other problem is VLAN-s. I must regonfiguse all VLANs before. So, vCenter and ESXi must not connect each other directly, but only through those firewalls chain. I just experiment this with two Kerio Control (I like Kerio Control because its management is very easy and comfort), but unsuccessful, no connection appears between ESXi and vCenter.  My VLAN is set up with ingress and egress run in different VLAN-s. I suspect I configured L2 only in one side and packets from other side mess something. In theory, if I want to broke connection, there is enough to disconnect only ingress or egress. But seems VMWare wasnt agreed with this. Its uncomfort to make much changes to physical switch VLAN configuration.

Reply
0 Kudos
SureshKumarMuth
Commander
Commander

There is no requirement (same network) as such for ESXi - VC communication provided if we have proper routing and port connections (902, 443 etc) which are needed for ESXi to communicate with VC. Also if you are using NATing between VC and ESXi it will not work and not supported by VMware as well. We have a server which is still isolated just because the host IP under NAT so that VC could not communicate with it. Though we do not have clear documentation on it, I personally feel the VC - ESXi connection is not just a connectivity via IP, there is an agent running on ESXi which is needed for VC-ESXi connectivity and config files on ESXi has the VC IP details, if there is a NATing, agent might be confused or there could be special config needed to make it work. Agent is playing important role in VC - ESXi communication as it continuously transferring host data to VC for multiple purpose (monitoring data, VC to ESXi commands etc ..). I may be wrong but I am assuming this based on my understanding . Lets see if we get more responses with different views on this query.

Regards,
Suresh
https://vconnectit.wordpress.com/
Reply
0 Kudos
IvarHome
Hot Shot
Hot Shot

>>>>>Also if you are using NATing between VC and ESXi it will not work and not supported by VMware as well.

So, and why? Why NAT dont work with VMWare? Have VMWare invent something new in networking? Its intersestin and why vmware dont publish this discovery? When NAT dont work, this mean there is still same subnet requirements and things dont work at all in L3 level but instead in L2 level.

>>>I personally feel the VC - ESXi connection is not just a connectivity via IP

Yes, I feel the same and this is bad or uncompleted developing practice by vmware. When they want to make our software only by L2, then say it also to public.

Ok, now I understand what you mean. But I talked about different kind of NAT-ing - when ESXi sees other side (firewall) as vCenter IP and vCenter sees ESXi with ESXi IP-s. When I have 2 firewalls with both SNAT and DNAT, then vCenter seest it communicate with ESXi IP, altough in reality its firewall and vice versa.

Reply
0 Kudos
sk84
Expert
Expert

I dont remenber where, I readed it many times.

Please show me one example. Because it's wrong. At least I know that already in version 4.1 different subnets for vCenter and ESXi hosts were supported. And this hasn't changed in all 5.x and 6.x versions.

Whole VMware documentation is one piece of shit, its absolutely useless.

That's your personal opinion. I know of only a few products that are so complex and offer such extensive documentation.

Both firewalls make SNAT and DNAT. So, ESXi and vCenter sees each other as in the same subnet, both firewalls uplinks looks like they are ESXi and vCenter IP-s, but I can configuse DNAT to connect also with any IP in ESXi.

As mentioned from others, this configuration is not supported and may also be a reason why you have some problems with your network and NSX. And I know from your other discussions that you don't care and you think VMware is stupid that they don't consider such an obvious scenario. But it is what it is.

--- Regards, Sebastian VCP6.5-DCV // VCP7-CMA // vSAN 2017 Specialist Please mark this answer as 'helpful' or 'correct' if you think your question has been answered correctly.
IvarHome
Hot Shot
Hot Shot

>>>That's your personal opinion. I know of only a few products that are so complex and offer such extensive documentation.

Of course its my opinion, I dont talk about some other person opinions.

>>>As mentioned from others, this configuration is not supported and may also be a reason why you have some problems with your network and NSX. And I know from your other discussions that you don't care and you think VMware is stupid that they don't consider such an obvious scenario. But it is what it is.

So, what configuration exactly you mean?

Reply
0 Kudos
sk84
Expert
Expert

So, what configuration exactly you mean?

This:

Both firewalls make SNAT and DNAT. So, ESXi and vCenter sees each other as in the same subnet,

NAT'ing between ESXi hosts and vCenter is not a supported configuration.

So, and why? Why NAT dont work with VMWare? Have VMWare invent something new in networking? Its intersestin and why vmware dont publish this discovery? When NAT dont work, this mean there is still same subnet requirements and things dont work at all in L3 level but instead in L2 level.

And since you have already shared your opinion on this, it still remains as it is. This configuration is simply not supported by VMware. Either because they simply don't like it or because there is a technical reason. We don't know, but there is nothing to discuss here. And they do not have to explicitly exclude every conceivable idea and technical possible setup. It's enough to just publish informations about supported setups and to say that only these setups are supported. And that's exactly what they do.

For example here:

https://docs.vmware.com/en/VMware-vSphere/6.7/vsphere-esxi-vcenter-server-67-networking-guide.pdf

Performance Best Practices for VMware vSphere 6.5

Storage and Networking

https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/support/landing-pages/virtual-supp...

VMware Knowledge Base

Or in the VVDs:

Physical Networking Design

Virtualization Network Design

In addition, this knowledge is taught in various courses about vSphere.

--- Regards, Sebastian VCP6.5-DCV // VCP7-CMA // vSAN 2017 Specialist Please mark this answer as 'helpful' or 'correct' if you think your question has been answered correctly.
Reply
0 Kudos
IvarHome
Hot Shot
Hot Shot

So, what configuration exactly you mean?

This:

Both firewalls make SNAT and DNAT. So, ESXi and vCenter sees each other as in the same subnet,

NAT'ing between ESXi hosts and vCenter is not a supported configuration.

And you want to say Vmware writes somewhere "when ESXi and vCenter see each other in same subnet, then this is not supported"? Are you sure. I still ask what is not supported? NAT? Network address translation? But when you translate addresses and then translate back? Just about this I was talking. Vmware wasnt sayed double translation is not supported also. Do you know what is NAT?

Do this "paper official" from vmware at all know what is dnat and snat? How vmware software finds out that I have there between some NAT. Its like little man-of-middle. Are you sure vmware software dont work there? Im pretty sure it works. Its not enough to say something is not supported when you have no idea about what talk is about.

Reply
0 Kudos
ThompsG
Virtuoso
Virtuoso

Here is a statement about using NAT between ESXi hosts and vCenter from VMware: https://kb.vmware.com/s/article/1010652

Reply
0 Kudos
IvarHome
Hot Shot
Hot Shot

>>>>Here is a statement about using NAT between ESXi hosts and vCenter from VMware: https://kb.vmware.com/s/article/1010652

I dont see here any restriction to my configuration. So, I think you just dont get it and I try to explain:

ESXi  ip     - 192.168.12.43

vCenter ip - 192.168.12.20

vCenter (source:192.168.12.20 dest:192.168.12.43) <====> DNAT (192.168.12.43-->192.168.50.2) / SNAT (192.168.12.20-->192.168.50.1)<----->DNAT (192.168.50.2-->192.168.12.43) / SNAT

(192.168.50.1-->192.168.12.20) <=====> ESXi (192.168.12.43)

Here we see 2 firewalls, both make DNAT and SNAT.  FW1 have interfaces 192.168.12.43 (the same as in ESXi) and 192.168.50.1  FW2 have interfaces 192.168.50.2 and 192.168.12.20 (the same as vCenter). This example dont do nothing useful, just 1:1  but now I show how it does something useful. Example in ESXi I have another vmkernel adapter with IP 192.168.12.59 and my first vmkernel becomes for some reason unaccessible (vlan or no teaming uplink):

vCenter <====> DNAT (192.168.12.43-->192.168.50.2) / SNAT (192.168.12.20-->192.168.50.1)<----->DNAT (192.168.50.2-->192.168.12.59) / SNAT

(192.168.50.1-->192.168.12.20) <=====> ESXi (192.168.12.43)

And connection from ESXi to vCenter is exactly vice versa, I wasnt painted it here. As you see ESXi sees all the time vCenter IP address. ESXi sees connection comes from vCenter, but actually its come from firewall.

Reply
0 Kudos
sk84
Expert
Expert

I really don't know what you can't understand about that. It is clearly formulated:

Using NAT between the vCenter Server system and ESXi/ESX hosts is an unsupported configuration and there is no workaround.

Maybe to make it clearer: Any kind of NAT between vCenter Server and ESXi hosts is not supported. It doesn't matter if it's a source nat or destination nat or both. It is not supported.

If you don't want to understand, I can't help you either. The term "supported" at this point means that a component or configuration has been tested by VMware and VMware can guarantees that it will work and that no problems with it should occur. So if you do something that is not supported by VMware, it can work, but it can also cause strange behavior. But then it's not VMware's fault, as you keep claiming, because you're doing something that isn't intended. So simple.

And as a last word from my side:

I have had these unnecessary discussions with you several times and I know that there is no point in discussing with you. You just want to do what comes into your mind and think you are right, no matter if it is intended or supported by the manufacturer or not. To make a real world comparison: You buy a street car, tinker with it, drive rallies and complain to the manufacturer that the car is broken and useless afterwards. Very useful...

--- Regards, Sebastian VCP6.5-DCV // VCP7-CMA // vSAN 2017 Specialist Please mark this answer as 'helpful' or 'correct' if you think your question has been answered correctly.
Reply
0 Kudos
IvarHome
Hot Shot
Hot Shot

sk84,  you talk not at all about what is in topic here. You talk about some other requirements, not those are here. Check my painted network topology again. Or maybe you dont know at all what is DNAT and SNAT.

Reply
0 Kudos