Can vCloud Director send TCP keepalives

[Theovd]

Hi,

I have setup vDirector in a seperate network zone than vCenter (and Chargeback, Oracle database and vShield Manager). This means that the connections between VCD and vCenter are routed over a firewall. What I see is that when VCD was idle for some hours, the first action (e.g. create VM) fails because the firewall has dropped the connection between VCD and vCenter.

A way to resolve this, is to let VCD send TCP keepalives. Is there a way of configuring this?

Regards,

Theo.

[admin]

Quick questions:

1. Are you seeing a "Connection Reset" error in vCD?

2. What is the timeout value of the firewall dropping the connections? What is its connection drop policy? Only idle connections?

3. What is the value of vCenter --> Administration --> vCenter Server Settings --> Advanced Settings --> vpxd.httpClientIdleTimeout?

[Theovd]

Hi,

And some quick answers:

Ad 1: No. On the first attempt after the timeout, vCloud Director tries it several times but does not get a response. I think that our firewall is configured to silently drop the traffic.

Ad 2: It drops connections after 3 hours of being idle.

Ad3: 900 (seconds). Quite strange as I see that connection is established longer. Does this setting also applies against web services (I assume that VCD invkoes vCenter's web services).

Theo.

[admin · Accepted solution]

More:

1. You mentioned you get a timeout on the first attempt, what exact error do you see? Could you give a screenshot?

2. The setting vpxd.httpClientIdleTimeout is only applicable to vCenter <-> ESX connections. Unfortunately, that won't help here.

3. Would you mind sharing the network topology of the components involved? - load balancer, vCD cells/DB, firewall, vCenter, ESX and potentially any other servers that are involved?

To answer your original question: No, as of vCD 1.0, it is not possible for vCD to send TCP keepalives. You may want to contact VMware Support if you're observing constant issues with this.

[Theovd]

Hi,

I already filed a SR and got replied that vCloud Redirector does not support TCP keepalives in this release. Our solution would be either:

• Configure the firewall in a way it does not close the connection

• Move vCloud Director from the DMZ to the same zone where vCenter is located and install a (apache) reversed proxy between the internet and vCloud Director.

I have to make up my mind on this. Has anyone experience with either solution?

Theo

[admin]

Most firewalls also support exclusion rules - you could add rules to exclude killing idle connections from the vCloud cells.

[Theovd]

Hi,

I configured the firewall (Cisco ASA 5520) that it uses 'Dead connection detection" for the VCD server. This resolved my problem. Such a config looks like:

access-list vdirector extended permit ip 10.16.32.10 255.255.255.255 any

class-map VDIRECTOR

match access-list vdirector

policy-map VDIRECTOR

    class VDIRECTOR

    set connection timeout tcp 2:00:00 half-closed 00:10:00 dcd 00:10:00 3

service-policy VDIRECTOR interface vlan8

1. First, I defined an access list containing my VCD server.
2. Defined a class map using the access list and uses this class to define a policy that after 2 hours sends max 3 dead connection detection (DCD) probes to the ends of the connections. When a reply is received, the firewall defines the connection as alive and resets the idle counter.
3. The service-policy definition effectuates the policy on the interface 'vlan8' (the interface the connections from the VCD server enters the firewall)

With this config, the firewall probes both ends of the connection while they are still alive in stead of just dropping the connection after a certain of idle time.

If there are more firewalls on the path from the VCD server to vCenter, there is no need to reconfigure as long as the timeout in the created policy is shorter than on the other ones. The probes sent by the reconfigured firewall will also reset the idle time counter for the connection on the other firewall.

Theo.

[Theovd]

Just got a reply from VMware that the issue will be solved in a next release. VMware did not gave the timing for the next release.

Theo