VMware

This Question is Answered

1 2 3 Previous Next 30 Replies Last post: Sep 28, 2009 12:22 PM by livinma1   Go to original post

Re: Renaming VMnics to match PCI Slot Numbers

15. Mar 21, 2009 6:39 PM in response to: aenagy
Click to view RichD's profile Novice 17 posts since
May 4, 2008
Haven't tested it at all, so can't say.

-Rich

Re: Renaming VMnics to match PCI Slot Numbers

16. Mar 21, 2009 6:57 PM in response to: aenagy
Click to view linuxdynasty's profile Enthusiast 26 posts since
May 16, 2008
I have not tested this script on ESXi, so I really can not say. But if you have issues with the vmnic names being out of order with the PCI ID's then I would assume it should work as well.

Re: Renaming VMnics to match PCI Slot Numbers

17. Mar 22, 2009 6:46 AM in response to: aenagy
Click to view dconvery's profile Virtuoso 1,885 posts since
May 10, 2006
aenagy wrote:
Will this script work on ESXi via RCLI? (Obviously not for the vswif NICs ;) )

I don't think this will work using the RCLI because of the way the script works. It needs to be executed locally, I believe. Although I haven't tested it, I would assume it would work by entering "unsupported mode" (also sometimes referred to as "Tech support mode") and running it from there.


If you DO run it from the unsupported mode on the ESXi server, it WILL change vswif0 as well.


After running the script, a reboot is required. The vswif0 reference could now potentially cause a disconnect if your vswif0 is not pointing to the first NIC on the PCI bus.


The intent here is for scripted installations where the NIC order is not what is expected and needs to be reset. I have seen this primarily on HP servers. It could be differences in the way isolinux kernel and the hypervisor kernel handles the PCI addresses.


Dave Convery
VMware vExpert 2009
http://www.dailyhypervisor.com

Careful. We don't want to learn from this.
Bill Watterson, "Calvin and Hobbes"

Re: Renaming VMnics to match PCI Slot Numbers

18. Mar 22, 2009 6:57 AM in response to: dconvery
Click to view tom howarth's profile Guru 7,322 posts since
Jul 25, 2005
It is not a just HP servers that suffer from it, Dell , and Fujitsu Seimens server do as well, I have not really found it to be an issue with HP servers, but then I could have been lucky.

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

Tom Howarth VCP / vExpert
VMware Communities User Moderator
Blog: www.planetvm.net
Contributing author for the upcoming book "VMware Virtual Infrastructure Security: Securing ESX and the Virtual Environment”.

Re: Renaming VMnics to match PCI Slot Numbers

19. Mar 22, 2009 11:39 AM in response to: tom howarth
Click to view dconvery's profile Virtuoso 1,885 posts since
May 10, 2006
You're right Tom. It is weird too. I have it with the DL580 servers, but blades are OK. I did a deployment recently with 16 Dell R900 servers and had no problem with it. I include itin all deployments just in case now anyway. I think it has something to do with the way the kernel treats the addressing of internal components vs. slot components.

Dave Convery
VMware vExpert 2009
http://www.dailyhypervisor.com

Careful. We don't want to learn from this.
Bill Watterson, "Calvin and Hobbes"

Re: Renaming VMnics to match PCI Slot Numbers

20. Apr 14, 2009 5:25 AM in response to: RichD
Click to view ksf's profile Novice 9 posts since
Aug 12, 2008
Has anyone bothered with updating the modules.conf as well?

If so, has anyone had success with the make-modules_conf script(?), I just got a load of syntax errors when trying this...

Re: Renaming VMnics to match PCI Slot Numbers

21. Apr 14, 2009 8:25 AM in response to: ksf
Click to view RichD's profile Novice 17 posts since
May 4, 2008

You need to fix both esx.conf and modules.conf in order to complete the fix properly.

I used the attached scripts to achieve that.

If you run into problems, respond to thread and I'll post an update.

-
Richard Dagenais, VCP

Attachments:

Re: Renaming VMnics to match PCI Slot Numbers

22. Apr 14, 2009 9:03 AM in response to: RichD
Click to view linuxdynasty's profile Enthusiast 26 posts since
May 16, 2008
Hey RichD I know in the past we had our little tit for tat about the modules.conf on esx3.5, but You are the first person who I have heard mention that they needed to change modules.conf and use my script at the same time.

Dconvery do you have to modify the modules.conf when you run my script http://www.linuxdynasty.org/View-details/Linux-Dynasty-Scripts-and-Programs/1-Vmware-ESX-NIC-Reordering-FIX-Script.html as well??
Till this day I have not have to modify modules.conf with 4+ nics running in my environment.

Re: Renaming VMnics to match PCI Slot Numbers

23. Apr 14, 2009 10:29 AM in response to: linuxdynasty
Click to view RichD's profile Novice 17 posts since
May 4, 2008

It's funny that you say that because the impression I got is that you kinda resented that I posted an alternate solution to the problem.

I think its quite cool that you posted your script online, and that you've been actively maintaining it, and I've given you credit for that since my first post.

There are two reasons why I started from scratch on this problem after trying your much earlier script in the lab.

First, you used python which was not supported by any of our resources in-house. We decided to leverage bash, gawk/awk, sed - tools that were very supportable and consistent with the rest of the deployment script suite we were developing. Your original version was not working correctly in our environment and we couldn't debug it, so we figured the problem out on our own and wrote our own script.

Also, you ignored the modules.conf file which is necessary to complete the nic renumeration properly. Otherwise you are going to have a port assignment mismatch in linux boot mode and lose connectivity. Further, you would leave the system inconsistent with the way it is intended to be deployed by VMware and when you do a support call and run into that problem, support will poo-poo the script.

Since I am a VCP and my primary mandate is to make things as conform to standards of practice as possible, I could not leave out the modules.conf fix from my script suite. If you look at the code I shared here, you'll see that the modules.conf fix is quite a bit more fancy that the esx.conf renumeration script. It had to be, to interpret the driver tables and pick the correct one.

The script was made by myself and another expert; combined we have over 30 years of Unix experience. We felt strongly that this was a required part of the solution and spent the extra time for good reason. The client (Federal Gov't) agreed. We wanted a reliable fix, not a partial hack.

An important note here is that there is no reason why you can't use my modules.conf fix script with your own python script. The two stand alone.

I don't have a website which I am driving traffic to with this thread. I published the scripts on the VMware community to help out my colleagues, not to compete with you. I think there is plenty of room for different solutions to this problem. In fact, I think that having a little "competition" of sorts has driven you to improve your script greatly which can only be a positive thing all around for those who choose to use it.

Cheers!

-Richard Dagenais, VCP

Re: Renaming VMnics to match PCI Slot Numbers

24. Apr 14, 2009 12:10 PM in response to: RichD
Click to view linuxdynasty's profile Enthusiast 26 posts since
May 16, 2008
WoW, you really went there, as professional as possible you are throwing your title around as well as your years of experience.... All I have been asking since the beginning of this thread and the other threads is WHY do you have to modify modules.conf. And I am not looking for because that is the VMware standard. Technically WHY??? The only reason I am harping on this, is because the Linux Kernel will assign the NIC's in the proper order based on PCI ID. Then after you are done with the ESX install, ESX will reorder them based on what ever criteria they are using. So All my script is doing is reordering back in the correct order based on PCI ID. Which you will never have to do again. So my question stands.. WHY do you HAVE to modify modules.conf??

I do not resent that you have another script out there. I really do not, as I am not in competition with you or anyone else, I just love giving back to the community. That is the sole purpose of my website! I just do not understand why you "HAVE" to modify modules.conf..... Let me ellaborate further.... You are the 1st user who has told me you "HAVE" to modify modules.conf and made it mandatory in all of your postings. I have yet to modify modules.conf with HP boxes and successfully pxe kicked my esx host's and yes my script has been used in 100's of esx deployments. So that is the only reason for me going back and forth with you. This is the first time that you say there is no reason not to modify modules.conf.


And you really do not want to make this posting about Experience and throwing around titles, because I can embarrass Admins who have been doing this type of work 3 to 4 longer then I have.

Re: Renaming VMnics to match PCI Slot Numbers

25. Apr 14, 2009 1:04 PM in response to: linuxdynasty
Click to view dconvery's profile Virtuoso 1,885 posts since
May 10, 2006
Looking at a modules.conf file from a recent engagement, it looks like it MAY need to be edited. I am not sure if kudzu discovers the changes are re-orders the modules.conf file. I would have to try it the next time I come into the need to give a definite answer. I CAN say, however, that on the last engagement where I used it with different NICs, the server ran fine. I do not have a modules.conf file available from one where I actually ran the script. Like I said, I would assume that kudzu would handle the changes to modules.conf. Here's what I did last time:

kickstart install
reboot
ran python script on ESX console
reboot
Kudzu detects changes during boot up and reboots again
system comes up fine

Maybe someone can confirm this on their system? The latest engagement I am working is ALL e1000 NICs, so modules.conf does not NEED to be edited

Here's an output from a modules.conf file from an HP:
alias eth0 e1000
alias eth1 e1000
alias eth2 e1000
alias eth3 e1000
alias eth4 e1000
alias eth5 e1000
alias eth6 e1000
alias eth7 e1000
alias eth8 e1000
alias eth9 e1000
alias eth10 e1000
alias eth11 e1000
alias eth12 e1000
alias eth13 e1000
alias eth14 e1000
alias eth15 e1000
alias eth16 bnx2
alias eth17 bnx2


Dave Convery
VMware vExpert 2009
http://www.dailyhypervisor.com
http://communities.vmware.com/servlet/JiveServlet/download/38-20623/vExpert_logo_100x57.jpg
Careful. We don't want to learn from this.
Bill Watterson, "Calvin and Hobbes"

Re: Renaming VMnics to match PCI Slot Numbers

26. Apr 14, 2009 1:06 PM in response to: dconvery
Click to view dconvery's profile Virtuoso 1,885 posts since
May 10, 2006
I just thought of something when I looked at my post...The bnx aliases point to the LAST NICs even though they are the on-boards. The python script was not run on this server.


Dave Convery
VMware vExpert 2009
http://www.dailyhypervisor.com
http://communities.vmware.com/servlet/JiveServlet/download/38-20623/vExpert_logo_100x57.jpg
Careful. We don't want to learn from this.
Bill Watterson, "Calvin and Hobbes"

Re: Renaming VMnics to match PCI Slot Numbers

27. Apr 14, 2009 1:37 PM in response to: dconvery
Click to view RichD's profile Novice 17 posts since
May 4, 2008

The bnx aliases point to the LAST NICs even though they are the on-boards.

It's important to note that modules.conf and esx.conf are two different problems, for two different boot states/kernels.

It's also important to note that the kernel in the installer has the same problem with NIC ordering, but that this is unique to that kernel and the driver loading order on the install image.

So there are three places you need to implement a workaround:

  • By selecting a different adapter during kickstart;

  • by re-ordering NICs during POST using either of the scripts in this thread, and;

  • by reordering modules.conf.

We wrote the make-modules_conf script to address the latter, to get the bnx's in their appropriate spot, and this consistent with nic renumeration of esx.conf (i.e. always in PCI address order, consistent with Best Practices).

make-modules_conf achieves this by parsing the PCI IDs, the driver tables provided by vmware, and building modules.conf from scratch.

I'm not sure why I'm getting flamed for pointing this out, but modules.conf needs to be fixed in order for the nic re-ordering to be complete. Of course fixing esx.conf will resolve nic ordering while running vmnix, and that is what matters 99.9% of the time. But if you reboot in linux mode you lose connectivity. make-modules_conf is the only script I am aware of that currently fixes this issue, which is why I wanted to share it with the class.

-Richard Dagenais, VCP

Re: Renaming VMnics to match PCI Slot Numbers

28. Apr 14, 2009 2:21 PM in response to: RichD
Click to view linuxdynasty's profile Enthusiast 26 posts since
May 16, 2008
that's all I was asking for. I just wanted to know why you have to set modules.conf. and what you are saying does makes sense. I never had to boot into linux mode on any of my esx servers so I never ran into this issue. the reason for the flaming is that you made this personal by implying that I am soliciting my site and throwing your title around as well. all I wanted to know is why and now that you explained it to me. I will research this issue. fyi... I run my site out of my own pocket and the sole purpose of the site is to share my knowledge with every one..

Re: Renaming VMnics to match PCI Slot Numbers

29. Aug 27, 2009 8:55 AM in response to: dconvery
Click to view keinstein's profile Lurker 2 posts since
Aug 26, 2009

Hello,

I downloaded the script and used it, it works great, however since i'm new to the whole ks and post script arena i would like to know how could i include this script in the %post section of my ks. I need to deploy more than 40 hosts in one night and i don't want to configure them manually. i jsut want to automate as much as possible. Thanks for your help!!

-- Ken

VMware Developer

SDKs, APIs, Videos, Learn and much more in the Developer community.

Learn More

Developer Sample Code

Increase your developer productivity with VMware API sample code.

Learn More

VMworld Sessions & Labs

Online access to the latest VMworld Sessions & Labs and online services.

Learn more

Purchase PSO Credits Online

Purchase credits to redeem training and consulting services online.

Buy Now

Community Hardware Software

View reported configurations or report your own.

Learn More

VMware vSphere

Come witness the next giant leap in virtualization.

Register Today

Communities