Red Hat Enterprise Linux 6 - Guest Cloning Problem

[jmgriffes]

I was recently assigned to work on a project to do with TSM (Tivoli Storage Management from IBM) and discovered a neat little problem with RHEL 6.

I can create a VM and install RHEL6 just fine. I was able to install VMware tools after some Google footwork, and all is working as it should (we got the assignment done). I was about to do some further testing to refine the documentation we created about TSM, and cloned one of the RHEL6 machines I had created.

Upon starting the new VM I noticed that VMware Tools reported no IP addresses. Looking at "ifconfig" shows only "lo" - No eth0 as there was on the original box. "ifup eth0" claims that the device is not present (double and triple-checked that the vNic is still there). Running "setup" shows eth0 still present, with one change. In the original build eth0 shows as "eth0 (eth0) - VMware VMXNET3 Ethernet Controller" in the clone it shows as "eth0 (eth0) - Ethernet" (this is duplicatable on the original machine by removing the vNIC, booting up, powering down and adding it again).

Has anyone had similar issues? RHEL5 duplicates just fine keeping its original vNIC intact, but I can't seem to get RHEL6 working with that.

Any help or suggestions would be great.

We're running ESXi 4.1

[markdjones82]

Jm, for whatever reason it is creating a new MAC address etc for the device.

So, to fix this go into the /etc/udev/70-peristent-net.rules and change eth1 to eth0.

Comment out the first entry.  Also, remove or set the proper HWADDR in your ifcfg-eth0 configuration.

All of this is a workaround, but it will get you up and running.

[jmgriffes]

Mark,

Thanks for the response. It didn't quite have everything I needed, but it gave me the push I needed to be able to get everything fixed. Here's the instructions I followed to get it working (though it is quite the workaround, hopefully VMware can get this fixed).

1. Go to /etc/udev/rules.d/

2. Open 70-persistent-net.rules with your favorite editor (I prefer "nano")

     There should be two devices listed starting with "SUBSYSTEM" (the same machine before cloning had only one)

3. Comment out the first line starting with "SUBSYSTEM" (a "#" will comment it out)

4. Go to the end of the second line (uncommented one) starting with "SUBSYSTEM" and change "eth1" to "eth0"

     Record the new MAC Address of the entry

5. Go to /etc/sysconfig/networking/devices

6. Open ifcfg-eth0 with your favorite editor

7. Change "HWADDR" to the new MAC Address you recorded

8. Change the IP while you're in here

9. Reboot the box, if ifcfg-eth0 is set to come up on boot, the vNIC should be working now.

Thanks again Mark for pointing me in the right direction to get this figured out.

[markdjones82]

No problem, I did write up my response in a quick hurry without all the proper steps.  Thank you for documenting it all.

I also have noticed issues with RHEL6 guests.  It looks as though I will stick with RHEL5 mostly until vmware catches up with it.

I also am unable to customize a RHEL6 guest

[jmgriffes]

No worries. It was very helpful.

Same problem with no customization of RHEL6. Are you on ESX/ESXi 4.1 Update 1? I'm curious if that changes anything.

[markdjones82]

No, we are still on 4.0 U2 at the moment.  I'd really like to hear more from vmware on this.

I also cannot get RHEL 6 to install the vmwaretools using the new Yum method until 4.1.

I think my plan of staying with RHEL5 for the near future is a good idea for now.

[sis9014]

Hi jmgriffes

I am on ESX 4.1 Update 1 and it is the same. still have to change it to RHEL 5 to be able to deploy VM.

Simon

[jmgriffes]

Thanks Simon,

Good to know that it isn't quite supported yet.

Thanks again!

Jared

[looprock]

This only assumes a single NIC but it'll get you through for testing:

I created a VM and just before converting it to a template which uses DHCP to get an address, I added this script to rc.local:

/root/bin/vmwarenetrewrite:
#! /bin/sh

FILE="70-persistent-net.rules"
SRC="/etc/udev/rules.d/${FILE}"
TMP="/tmp/${FILE}.tmp"
SRCIF="/etc/sysconfig/network-scripts/ifcfg-eth0"
TMPIF="/tmp/ifcfg-eth0.tmp"
SRCRC="/etc/rc.d/rc.local"
TMPRC="/tmp/rc.local.tmp"

# comment out all subbsystem lines
/bin/cat ${SRC} |/bin/sed "s/SUBSYSTEM/#SUBSYSTEM/g" > ${TMP}
# uncomment the last entry and change eth1 to eth0
lines=`/bin/cat ${TMP}|/usr/bin/wc -l`
min=$((lines - 1))
/bin/cat ${TMP} |/usr/bin/head -${min} > ${SRC}
/bin/cat ${TMP} |/usr/bin/tail -1 | /bin/cut -c 2- |/bin/sed 's/eth1/eth0/'>> ${SRC}
# modify ifcfg-eth0 with correct MAC address
MACADDR=`/bin/cat ${SRC} |/usr/bin/tail -1|/bin/awk -F'"' {'print $8'}`
/bin/cp -a ${SRCIF} ${TMPIF}
OLDLINE=`/bin/grep "HWADDR" ${SRCIF}`
/bin/cat ${TMPIF} |/bin/sed "s/${OLDLINE}/HWADDR=${MACADDR}/" > ${SRCIF}
# remove this script from rc.local
/bin/cp -a ${SRCRC} ${TMPRC}
/bin/cat ${TMPRC} |grep -v "vmwarenetrewrite" > ${SRCRC}
#/bin/rm -f ${TMP} ${TMPIF} ${TMPSRC}
# reboot for changes to take effect
/sbin/shutdown -r now

[AndyR8939]

Thanks for this guys, really helped me out when doing some RHEL6 clones on ESXi5.

I had to adapt jmgriffes steps though as I needed a few more steps and had different loctions for some reason.  This is what I did.

1. Go to /etc/udev/rules.d/
2. Open 70-persistent-net.rules with vi.
3. Comment out the first line starting with "SUBSYSTEM"- should be eth0
4. Go to the end of the second line (uncommented one) starting with "SUBSYSTEM" and change "eth1" to "eth0".  Make sure Mac address is the correct new one.
5. Go to /etc/sysconfig/network-scripts
6. cp ifcfg-eth1 to ifcfg-eth0
7. rm ifcfg-eth1
8. Open ifcfg-eth0 with vi
9. Change NAME to eth0
10. Change DEVICE to eth0
11. Change "HWADDR" to the new MAC Address if not correct
12. Reboot the box, if ifcfg-eth0 is set to come up on boot, the vNIC should be working now.

That fixed it on my VMs anyway.  Thanks guys!

[cyclooctane]

This problem effects all versions of redhat based distros that use udev.

And I have also had the problem on debian and suse based systems.

The same thing happens if you have a physical box and pull the NIC out and replace it with one of the above distros.

It is because the MAC address of the NIC is hard coded into the 70-persistant-net-rules file as was mentioned

earlier.

This is done to prevent any possibility of a race condition in a system with multiple NICs on start up.

(otherwise udev assigns eth0 to the first nic to come up, eth1 to the second and so on)

As you could probably see, this is a huge issue for linux systems that are used as firewall / gateway systems.

However for VMs that only have 1 nic it is a pain in the neck

For the VMs that I admin at work, (most of them have a single NIC) I have removed the HWADDR from ifcfg-eth0, and set a script that does an

rm of 70-persistant-net-rules on shutdown.

For Centos 5 and Centos 6, this workaround seems to work fine.

Regards

Cyclooctane