VMware

This Question is Answered

1 "correct" answer available (10 pts) 1 "helpful" answer available (6 pts)
1 2 Previous Next 18 Replies Last post: Nov 6, 2009 7:47 AM by matioswald  

How to force VCB backup using certain network interface posted: Nov 3, 2009 8:00 AM

Click to view matioswald's profile Novice
matioswald
10 posts since
Jun 18, 2009

Hello,

we have a ESX 3.5 einvironment wit three nodes and a NetApp storage behind.

All ESX nodes and also the backup server have two network interfaces, one on 100BT and one on GB.

We are backing up with BackupExec and VCB trying to force the traffic of the VCB backup between the ESX host and the backup server through the 1000BT interface, without success.

The ESX server receives the data through the 1000BT network, but sends it through the 100BT to the VCB/Backup server.

Is there a way to configure / force the network traffic between VCB and ESX ?

Thanks a lot for your help,

Matthias Oswald

Re: How to force VCB backup using certain network interface

1. Nov 3, 2009 8:09 AM in response to: matioswald
Click to view AndreTheGiant's profile Guru
AndreTheGiant
5,916 posts since
Aug 28, 2008
Do you have a COS on both network?
Also do you have a custom hosts file on VCB with "internal" IP?

Andre

Re: How to force VCB backup using certain network interface

2. Nov 3, 2009 2:48 PM in response to: matioswald
Click to view LEslinger's profile Enthusiast
LEslinger
29 posts since
Oct 30, 2009

Is this a dual homed computer to separate IP subnets?

Check the IP Routing Table and possibly rearrange your NIC binding order on the VCB server. You can also try adjusting the "Inteface metric" assigned to each NIC.

Dual homing is all about knowing how routing tables work and how the NICs are ordered.


Re: How to force VCB backup using certain network interface

3. Nov 4, 2009 2:32 AM in response to: LEslinger
Click to view matioswald's profile Novice
matioswald
10 posts since
Jun 18, 2009

It is not dual homed, we just want to use the GB interface for the data traffic to the SAN storage and the othet NIC for management.

The NIC binding order on the two WIndows boxes (Backup server, vCenter server) looks ok, the GB interface comes first.

The routing table looks also fine I guess, pinging the Backup server returns the 192.168.1 GB network:

root@vclnode03 vmware# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.2.0 * 255.255.255.0 U 0 0 0 vswif2
192.168.113.0 * 255.255.255.0 U 0 0 0 vswif0
192.168.1.0 * 255.255.255.0 U 0 0 0 vswif1
169.254.0.0 * 255.255.0.0 U 0 0 0 vswif2
default router-sw.snct. 0.0.0.0 UG 0 0 0 vswif0
root@vclnode03 vmware# ping srvbck01
PING srvbck01.snct.lux (192.168.1.220) 56(84) bytes of data.
64 bytes from srvbck01.snct.lux (192.168.1.220): icmp_seq=0 ttl=128 time=1.45 ms
64 bytes from srvbck01.snct.lux (192.168.1.220): icmp_seq=1 ttl=128 time=0.325 ms
64 bytes from srvbck01.snct.lux (192.168.1.220): icmp_seq=2 ttl=128 time=0.336 ms

      • srvbck01.snct.lux ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 0.325/0.704/1.451/0.528 ms, pipe 2
root@vclnode03 vmware# ping srvbck01

When looking at the network performance of the ESX server with vCenter client, I see incoming traffic from the SAN through the GB interface

but outgoing traffic to the backup server through the 100BT network.

Any other idea wherw I can look at ?

Thanks

Matthias

Re: How to force VCB backup using certain network interface

4. Nov 4, 2009 6:42 AM in response to: matioswald
Click to view LEslinger's profile Enthusiast
LEslinger
29 posts since
Oct 30, 2009

This appears to the route that the ESX server is issuing.

192.168.1.0 * 255.255.255.0 U 0 0 0 vswif1

What is the config of vswif1? Is it using the 1000BT or the 100BT? Or both? If it is both then configure the 100BT to be a standbay in the portgroup.

If it is the 100BT then you need to change the config to use the 1000BT.

Re: How to force VCB backup using certain network interface

5. Nov 4, 2009 7:27 AM in response to: LEslinger
Click to view matioswald's profile Novice
matioswald
10 posts since
Jun 18, 2009
vswif1 is a team of two NIC, both on 1000BT.

This interface on the ESC is used to receive the data from the storage, but the transmission to the backup server,

which has also has one 100BT and a team of two 1000BT, is going through the 100BT.


The VCB attacks the vCenter through the 1000BT and the ESX sends back to the VCB/Backup server on the 100BT.


I'm still not sure wehere the config problem is ?


ESX networking, VCB config, vCenter, Backup Exec config, host networking (host files, ???


Thanks for your support so far.

Re: How to force VCB backup using certain network interface

6. Nov 4, 2009 12:49 PM in response to: matioswald
Click to view LEslinger's profile Enthusiast
LEslinger
29 posts since
Oct 30, 2009

Is srvbck01.snct.lux (192.168.1.220) the ipaddress of the Teamed GB's or the the 100BT?

Do both interfaces have separate ip address registered as the same name in DNS? Does your DNS server do round robin?

If so try changing the 100BT interfact to something else like srvbck01B.snct.lux, or try configuring a host entry on the ESX server to point to the GB interfaces of the backup server.

If that isn't it provide the IP configuration for the backup server, routing table, and nic binding information of the backup server.

Re: How to force VCB backup using certain network interface

7. Nov 5, 2009 2:07 AM in response to: LEslinger
Click to view matioswald's profile Novice
matioswald
10 posts since
Jun 18, 2009
I launch the backup from the backup server with

vcbMounter.exe -h 192.168.1.221 -u snct\backup -p ??? -a ipaddr:192.168.1.102 -r f:\mnt\vmdbs02 -t fullvm -m nbd

-
Rgs,
Matthias
moswald@snct.lu
+352 357 214 341

Re: How to force VCB backup using certain network interface

8. Nov 5, 2009 2:17 AM in response to: LEslinger
Click to view matioswald's profile Novice
matioswald
10 posts since
Jun 18, 2009
Hi there,

thanks a lot for your efforts,

we noticed that both IPs of the srvbck01 192.168.113.189 (100BT) and 192.168.1.220 (GB) were in the DNS.
We changed the setting on the network and removed the GB from the DNS.

On the ESX server I added the GB interface in the hosts file
root@vclnode03 root# cat /etc/hosts
  1. Do not remove the following line, or various programs
  2. that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
192.168.113.188 vclnode03.snct.lux vclnode03
192.168.1.220 srvbck01.snct.lux srvbck01
root@vclnode03 root# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.2.0 * 255.255.255.0 U 0 0 0 vswif2
192.168.113.0 * 255.255.255.0 U 0 0 0 vswif0
192.168.1.0 * 255.255.255.0 U 0 0 0 vswif1
169.254.0.0 * 255.255.0.0 U 0 0 0 vswif2
default router-sw.snct. 0.0.0.0 UG 0 0 0 vswif0
root@vclnode03 root# ping srvbck01
PING srvbck01.snct.lux (192.168.1.220) 56(84) bytes of data.
64 bytes from srvbck01.snct.lux (192.168.1.220): icmp_seq=0 ttl=128 time=1.32 ms
64 bytes from srvbck01.snct.lux (192.168.1.220): icmp_seq=1 ttl=128 time=0.525 ms
root@vclnode03 root#

The backup server srvbck01 has a host file
127.0.0.1 localhost
#::1 localhost
127.0.0.1 srvbck01.snct.lux
192.168.1.221 srvvm01-san
192.168.1.213 vclnode03

The vCenter server srvvm01 has an empty host file
127.0.0.1 localhost
::1 localhost

Backup server configuration:
C:\Users\Backup>hostname
srvbck01

C:\Users\Backup>ipconfig

Windows IP Configuration

Ethernet adapter GBBCK:

Connection-specific DNS Suffix . :
IPv4 Address. . . . . . . . . . . : 192.168.1.220
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . :

Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . :
IPv4 Address. . . . . . . . . . . : 192.168.113.189
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 192.168.113.254

Ethernet adapter Local Area Connection 2:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :

Tunnel adapter Local Area Connection* 8:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :

Tunnel adapter Local Area Connection* 9:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :

Tunnel adapter Local Area Connection* 13:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :

Tunnel adapter Local Area Connection* 14:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :

C:\Users\Backup>route print
===========================================================================
Interface List
17 ...00 24 e8 37 10 95 ...... BASP Virtual Adapter
10 ...00 24 e8 37 10 91 ...... Broadcom BCM5709C NetXtreme II GigE (NDIS VBD Client)
11 ...00 24 e8 37 10 93 ...... Broadcom BCM5709C NetXtreme II GigE (NDIS VBD Client) #2
1 ........................... Software Loopback Interface 1
14 ...00 00 00 00 00 00 00 e0 isatap.{8E6AF430-DF3F-4580-A664-B50167CF2F67}
18 ...00 00 00 00 00 00 00 e0 isatap.{4DCFB2B1-5225-48BC-A8C0-C911F1DE046A}
15 ...02 00 54 55 4e 01 ...... Teredo Tunneling Pseudo-Interface
16 ...00 00 00 00 00 00 00 e0 isatap.{90BD23AA-B239-4011-B4C7-49C4F70C7931}
===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
0.0.0.0 0.0.0.0 192.168.113.254 192.168.113.189 276
127.0.0.0 255.0.0.0 On-link 127.0.0.1 306
127.0.0.1 255.255.255.255 On-link 127.0.0.1 306
127.255.255.255 255.255.255.255 On-link 127.0.0.1 306
192.168.1.0 255.255.255.0 On-link 192.168.1.220 266
192.168.1.220 255.255.255.255 On-link 192.168.1.220 266
192.168.1.255 255.255.255.255 On-link 192.168.1.220 266
192.168.113.0 255.255.255.0 On-link 192.168.113.189 276
192.168.113.189 255.255.255.255 On-link 192.168.113.189 276
192.168.113.255 255.255.255.255 On-link 192.168.113.189 276
224.0.0.0 240.0.0.0 On-link 127.0.0.1 306
224.0.0.0 240.0.0.0 On-link 192.168.1.220 266
224.0.0.0 240.0.0.0 On-link 192.168.113.189 276
255.255.255.255 255.255.255.255 On-link 127.0.0.1 306
255.255.255.255 255.255.255.255 On-link 192.168.1.220 266
255.255.255.255 255.255.255.255 On-link 192.168.113.189 276
===========================================================================
Persistent Routes:
Network Address Netmask Gateway Address Metric
0.0.0.0 0.0.0.0 192.168.113.254 Default
===========================================================================

IPv6 Route Table
===========================================================================
Active Routes:
If Metric Network Destination Gateway
1 306 ::1/128 On-link
1 306 ff00::/8 On-link
===========================================================================
Persistent Routes:
None

C:\Users\Backup>ping vclnode03

Pinging vclnode03 http://192.168.1.213 with 32 bytes of data:
Reply from 192.168.1.213: bytes=32 time<1ms TTL=64
Reply from 192.168.1.213: bytes=32 time<1ms TTL=64

Ping statistics for 192.168.1.213:
Packets: Sent = 2, Received = 2, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
Control-C
^C
C:\Users\Backup>

I checked again the binding order:

GB (team)
1000BT
1000BT
100BT

-
Rgs,
Matthias
moswald@snct.lu
+352 357 214 341

Re: How to force VCB backup using certain network interface

9. Nov 5, 2009 8:26 PM in response to: matioswald
Click to view LEslinger's profile Enthusiast
LEslinger
29 posts since
Oct 30, 2009

Ok, both of your servers are "Dual Homed" meaning they have connections to separate subnets 192.168.1.0 and 192.168.113.0 respectively.

It appears your ESX server may have two Service Consoles? One for each IP Subnet? 192.168.1.213 and 192.168.113.118 repectively.

If these are not both Service Consoles, then one must be a vmKernel and the other the SC. Please let me know what that configuration is.

Your ESX server Hosts file is using the IP address of 192.168.113.188 in the hosts file....which should correspond to a Service Console.

On both server the Default Route is the 113 subnet (100BT), and not the GB subnet.

Your Backup server is missing a Host entry for itself, you may want to put that in as the 1.220 IP Address.


Re: How to force VCB backup using certain network interface

10. Nov 5, 2009 8:32 PM in response to: LEslinger
Click to view LEslinger's profile Enthusiast
LEslinger
29 posts since
Oct 30, 2009

Previously you stated. The ESX server receives the data through the 1000BT network, but sends it through the 100BT to the VCB/Backup server.

How do you know this?

Assuming this is true, that would potentially mean (if my assumptions are correct) that the Service Console is responsible for sending data to the VCB server.

From my previous post, if you have two service consoles already configured, then configer the second SC as the "Default" OR

If you currently only have one SC, configure a second Service console on the GB interface, but keep the orginial as the "Default"


Re: How to force VCB backup using certain network interface

11. Nov 6, 2009 1:17 AM in response to: LEslinger
Click to view matioswald's profile Novice
matioswald
10 posts since
Jun 18, 2009
We see the network traffic in the vCenter clinet -> node -> Performance -> Network ...
During vcb backup the measurements on the GB RX is close to the 100BT TX and also
under the Windows Task Manager -> Networking on the backup server, only the 100BT is used.

Now after looking with esxcfg-route -l,
what strikes me is that the interface on the storage system (NetApp) is
actually defined as the default route, not the ESX GB.
This might be a mistake ???
Shouldn't it be the 1000BT of the ESX itself?
Is the default route actually affecting since the subnets should be properly defined ?

The Linux route command returns a port on the switch (100BT) as the default route.

-
Rgs,
Matthias
moswald@snct.lu
+352 357 214 341

Re: How to force VCB backup using certain network interface

12. Nov 6, 2009 4:39 AM in response to: matioswald
Click to view LEslinger's profile Enthusiast
LEslinger
29 posts since
Oct 30, 2009
In ESX the default gateway(s) can be defined on a Service Console or vmKernel Ports for Storage or vMotion. Which get used depends on the associated service using the IP. Make sure you have Default Gateways defined on both these configurations. Normally, you can only have one Defautl GW on a host, so I think the SC takes precendence...but not sure....your last post may contradict that.

This is also why i asked if you have two SC's or One Service Console and one vmKernel? Let me know which IP is which interface.

Your most recent post is telling me this...which is different than previusly thought.

1) The ESX Server is sending and receiving data via the GB

2) The Backup Sever is sending and recieving on the 100BT.

The issue appears to be the backup server.

What is the vcbMounter command you are using? Post the real IP Addresses in the command.

Re: How to force VCB backup using certain network interface

13. Nov 6, 2009 5:24 AM in response to: LEslinger
Click to view matioswald's profile Novice
matioswald
10 posts since
Jun 18, 2009

Sorry for the confusion.

The ESX server is receiving from the storage on the GB but sends to the backup server on 100BT.

The vcb command is

vcbMounter.exe -h vCenterServer -u backup -p ??? -a ipaddr:theVirtualMavhine IP -r f:\mnt\vmdbs02 -t fullvm -m nbd

root@vclnode03 root# esxcfg-nics -l
Name PCI Driver Link Speed Duplex MTU Description
vmnic3 0a:00.00 bnx2 Down 0Mbps Half 1500 Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T
vmnic6 24:00.00 e1000 Down 0Mbps Half 1500 Intel Corporation 82571EB Gigabit Ethernet Controller
vmnic8 25:00.00 e1000 Down 0Mbps Half 1500 Intel Corporation 82571EB Gigabit Ethernet Controller
vmnic9 25:00.01 e1000 Down 0Mbps Half 1500 Intel Corporation 82571EB Gigabit Ethernet Controller
vmnic0 03:00.00 bnx2 Up 100Mbps Full 1500 Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T
vmnic4 23:00.00 e1000 Up 1000Mbps Full 1500 Intel Corporation 82571EB Gigabit Ethernet Controller
vmnic5 23:00.01 e1000 Up 1000Mbps Full 1500 Intel Corporation 82571EB Gigabit Ethernet Controller
vmnic7 24:00.01 e1000 Down 0Mbps Half 1500 Intel Corporation 82571EB Gigabit Ethernet Controller
vmnic1 05:00.00 bnx2 Up 100Mbps Full 1500 Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T
vmnic2 08:00.00 bnx2 Down 0Mbps Half 1500 Broadcom Corporation Broadcom NetXtreme II BCM5708 1000Base-T
root@vclnode03 root# esxcfg-vswitch -l
Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch0 64 4 64 1500 vmnic0

PortGroup Name VLAN ID Used Ports Uplinks
VM Network 0 0 vmnic0
Service Console 0 1 vmnic0

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch1 64 8 64 1500 vmnic5,vmnic4

PortGroup Name VLAN ID Used Ports Uplinks
SAN Network 0 2 vmnic4,vmnic5
Servicekonsole ISCSI0 1 vmnic4,vmnic5
VMkernel ISCSI 0 1 vmnic4,vmnic5

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch2 64 5 64 1500 vmnic7

PortGroup Name VLAN ID Used Ports Uplinks
Servicekonsole 0 1 vmnic7
VMkernel VMotion 0 1 vmnic7

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch3 64 10 64 1500 vmnic2,vmnic1

PortGroup Name VLAN ID Used Ports Uplinks
Virtual Machine Network 1130 6 vmnic1,vmnic2

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch4 64 1 64 1500

PortGroup Name VLAN ID Used Ports Uplinks
Temp VSwitch 0 0

root@vclnode03 root# esxcfg-route -l

VMkernel Routes:
Network Netmask Gateway
192.168.1.0 255.255.255.0 Local Subnet "the GB network"

192.168.2.0 255.255.255.0 Local Subnet "the Vmotion subnet"
default 0.0.0.0 192.168.1.11 "the IP of the NetApp stoarge 1000BT interface" which is an interafce not on theESX

root@vclnode03 root#

comments in ""

Not sure what should be set in the esxcfg-route ?


Re: How to force VCB backup using certain network interface

14. Nov 6, 2009 6:07 AM in response to: matioswald
Click to view LEslinger's profile Enthusiast
LEslinger
29 posts since
Oct 30, 2009
Is the 192.168.1.0 subnet routable? Or is it just subnet used for the storage and backup?

I'm pretty sure your issue appears to be that a Service Console is not connected to the 192.168.1.0 subnet. As far as i know vcbMounter will only communicate with a Service Console IP.

Since the 192.168.113.0 network holds the SC, that would explain why ESX is sending data and the Backup Server is receiving data on that on those interfaces.

vcbMounter will communicate with VC server. What ever IP address (SC) is registered in VC will be the one that the vcbMounter uses.

If 192.168.1.0 is routable you should create a second SC on that network, then register the ESX host (disconnect change DNS entry to use 192..168.1.0 Ip Address and readd) to VC Server with the new IP configuration.

If 192.168.1.0 is not routable you shouldn't use the VC server to do the backup use the ESX host directly via a Second SC port on the 192.168.1.0 subnet.

Make sense?

Assuming 192.168.1.0 is not routable already.

A third option would be to dual home your VC server to the 192.168.1.0 subnet as well....and only register the 192.168.1.0 SC's in VC server.

A fouth option would be to dual home your VC server and drop you 100BT interfaces all together.

A fifth option would be make 192.168.1.0 routable with the rest of the network and drop the 100BT interfaces all together.

As for the default 0.0.0.0 192.168.1.11 "the IP of the NetApp stoarge 1000BT interface" which is an interafce not on theESX

I think you mistakenly added in the 192.168.1.11 ip address as the Default Gateway for the vmKernel storage network. If this is not routable you should remove the default gateway setting. If it is routable you should add in the correct router ip address.

1 2 Previous Next Go to original post

Actions

VMware Developer

SDKs, APIs, Videos, Learn and much more in the Developer community.

Learn More

Developer Sample Code

Increase your developer productivity with VMware API sample code.

Learn More

VMworld Sessions & Labs

Online access to the latest VMworld Sessions & Labs and online services.

Learn more

Purchase PSO Credits Online

Purchase credits to redeem training and consulting services online.

Buy Now

Community Hardware Software

View reported configurations or report your own.

Learn More

VMware vSphere

Come witness the next giant leap in virtualization.

Register Today

Communities