esstokes1
Enthusiast
Enthusiast

ESXi 4.1 scripted install

In my kickstart file I have the following two firstboot statements:

%firstboot --unsupported --interpreter=busybox --level=998

#Add pNIC vmnic1,vmnic2 and remove vmnic0 to vSwitch0

/usr/sbin/esxcfg-vswitch -L vmnic1 vSwitch0

/usr/sbin/esxcfg-vswitch -L vmnic2 vSwitch0

/usr/sbin/esxcfg-vswitch -U vmnic0 vSwitch0

  1. set multipath policy to round-robin for EMC arrays

/usr/sbin/esxcli nmp satp setdefaultpsp -s VMW_SATP_SYMM -P VMW_PSP_RR

/usr/sbin/esxcli nmp satp setdefaultpsp -s VMW_SATP_CX -P VMW_PSP_RR

echo "Configuring NTP"

echo "restrict default kod nomodify notrap noquerynopeer" > /etc/ntp.conf

echo "restrict 127.0.0.1" >> /etc/ntp.conf

echo "server xxx.xxx.xxx.xxx" >> /etc/ntp.conf

echo "driftfile /var/lib/ntp/drift" >> /etc/ntp.conf

/sbin/chkconfig --level 345 ntpd on

/etc/init.d/ntpd start

mkdir -p /var/updates ; cd /var/updates

wget http://xxx.xxx.xxx.xxx/os/esxi-4.1/updates/EMU-lpfc820-8.2.1.75.19-offline_bundle-274899.zip

wget http://xxx.xxx.xxx.xxx/esxi-4.1/updates/SVE-be2net-2.102.404.0-offline_bundle-281453.zip

/usr/bin/vim-cmd /hostsvc/maintenance_mode_enter

sleep 10

cd /var/updates

for zipfile in `ls *.zip` ; do

/usr/sbin/esxupdate --bundle=/var/updates/$ update

done

/usr/bin/vim-cmd /hostsvc/maintenance_mode_exit

sleep 10

/usr/sbin/esxcfg-advcfg -s 64 /Disk/SchedNumReqOutstanding

/usr/sbin/esxcfg-advcfg -s 1 /Disk/UseLunReset

/usr/sbin/esxcfg-advcfg -s 0 /Disk/UseDeviceReset

  1. Set NFS advanced Configuration Settings

/usr/sbin/esxcfg-advcfg -s 30 /Net/TcpipHeapSize

/usr/sbin/esxcfg-advcfg -s 120 /Net/TcpipHeapMax

/usr/sbin/esxcfg-advcfg -s 10 /NFS/HeartbeatMaxFailures

/usr/sbin/esxcfg-advcfg -s 12 /NFS/HeartbeatFrequency

/usr/sbin/esxcfg-advcfg -s 5 /NFS/HeartbeatTimeout

/usr/sbin/esxcfg-advcfg -s 64 /NFS/MaxVolumes

%firstboot --unsupported --interpreter=busybox --level=9999

/sbin/reboot

I can see the server run the 998.firstboot_001 script from the console and the server also runs the 9999.firstboot_002 script which performs a reboot. The problem is these scripts are never moved out of /etc/vmware/init/init.d/ so they are being executed on every reboot which means the server is constantly rebooting. Has anyone else seen this? Is there a better way to reboot the server (the reboot is required for the esxupdate).

Thanks!

0 Kudos
13 Replies
lamw
Community Manager
Community Manager

Instead of creating two custom startup scripts, just throw the "reboot" command at the end of your first script "998" and that will handle the reboot and remove the init script.

=========================================================================

William Lam

VMware vExpert 2009,2010

VMware scripts and resources at:

Twitter: @lamw

Getting Started with the vMA (tips/tricks)

Getting Started with the vSphere SDK for Perl

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

0 Kudos
esstokes1
Enthusiast
Enthusiast

That was my first attempt but I was seeing the same thing except 999.firstboot_license, 999.firstboot_password, and 999.firstboot_remove weren't ran ... that's why I added a second firstboot section that I knew would run last. Can you tell me what the difference is between /vmfs/volumes/Hypervisor1 and /vmfs/volumes/Hypervisor2? I'm thinking it has something to do with the boot.cfg not removing the onetime.tgz entry. I just tried removing it manually so we'll see what that does.

0 Kudos
lamw
Community Manager
Community Manager

Interesting, I currently have all my customization in firstboot which I set the level to 998 and I just have the very last line as a reboot and everything is handled correctly.

=========================================================================

William Lam

VMware vExpert 2009,2010

VMware scripts and resources at:

Twitter: @lamw

Getting Started with the vMA (tips/tricks)

Getting Started with the vSphere SDK for Perl

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

0 Kudos
Dave_Mishchenko
Immortal
Immortal

Did you check the logs to see why the remove script was not being run?

/vmfs/volumes/Hypervisor1 and /vmfs/volumes/Hypervisor2

These correspond with /bootbank and /altbootbank. /bootbank is the firmware that ESXi has booted from. When you patch ESXi you write a complete boot image to /altbootbank and then the system is set to subsequently boot from that partition. /altbootbank becomes the new /bootbank. The next patch repeats that process.

You might need to mimic some of the remove script actions to clean up the first boot scripts from /altbootbank.






Dave

VMware Communities User Moderator

Now available - vSphere Quick Start Guide

Do you have a system or PCI card working with VMDirectPath? Submit your specs to the Unofficial VMDirectPath HCL.

0 Kudos
GuinanP
Contributor
Contributor

Hi, I have seen exactly the same thing and the root cause was running the esxupdate command. From my testing, removing the esxupdate --bundle command should solve the issue of leaving behind the bootscripts causing the loop. It is almost as if the esxupdate process is somehow calhingwith the installer logic but not had time to investigate much further. I am looking at a workaround at the moment.

Regards Paul

0 Kudos
esstokes1
Enthusiast
Enthusiast

I was able to work around the problem by doing the following in my kickstart file:

%firstboot --unsupported --interpreter=busybox --level=9999

cp -f /vmfs/volumes/Hypervisor1/boot.cfg /vmfs/volumes/Hypervisor2/boot.cfg

sleep 300

/sbin/reboot

it looks like the 999.firstboot_remove was removing the onetime.tgz entry from the boot.cfg file in /vmfs/volumes/Hypervisor1 but not from /vmfs/volumes/Hypervisor2. Reading Dave's entry above, it looks like system creating an /altbootbank from Hypervisor2 because of the patching but the boot.cfg isn't being changed thus causing the problem. I added the 5 minute sleep to give the server a chance to clean-up and finish making the boot image .... I'm not sure this is needed.

0 Kudos
esstokes1
Enthusiast
Enthusiast

OK, one more note .... I had to copy /vmfs/volumes/Hypervisor2/* (/altbootbank) to /vmfs/volumes/Hypervisor1 (/bootbank) before the reboot so that the files updated by esxupdate would be seen. I have no idea how boot.cfg gets overwritten in Hypervisor1 but none of the other files are updated. I have a work around but if someone can shed some light on this then it would be much appreciated.

0 Kudos
lamw
Community Manager
Community Manager

I'm not sure if this will help, but what about moving the patching portion to secondary init script and set that to 9999 which will happen after the initial update of the onetime.tgz? This will ensure the host is ready to go prior to patching and then you can issue a reboot which hopefully will take care of all changes?

=========================================================================

William Lam

VMware vExpert 2009,2010

VMware scripts and resources at:

Twitter: @lamw

Getting Started with the vMA (tips/tricks)

Getting Started with the vSphere SDK for Perl

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

0 Kudos
lamw
Community Manager
Community Manager

I just confirmed by moving your patching to the very very end, you will not end up in a loop.

I just realized in my builds, I too have some vendor drivers that I needed to install and if I do it in the firstboot and then issue a reboot, the boot.cfg are not updated properly as you have noted.

I basically have two set of scripts, level 998 for all post-config and then level 9999 for patching and also containing the reboot command.

Give this a try and let me know if it works for you

=========================================================================

William Lam

VMware vExpert 2009,2010

VMware scripts and resources at:

Twitter: @lamw

Getting Started with the vMA (tips/tricks)

Getting Started with the vSphere SDK for Perl

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

0 Kudos
esstokes1
Enthusiast
Enthusiast

@lamw: i played with the order yesterday and found that moving the esxupdate into the second 'firstboot' file gives me what i was looking for. thanks for tinkering with with. also, thanks for the info at http://www.virtuallyghetto.com/2010/09/automating-esxi-41-kickstart-tips.html ... very helpful in getting me started down this road

0 Kudos
GuinanP
Contributor
Contributor

Hi

Thanks also for all the information regarding this issue. I noticed on http://www.virtuallyghetto.com/2010/09/automating-esxi-41-kickstart-tips.html you mentioned the issue with ntp requiring a reboot. I think you can get around this by putting the ntp config in a separate firstboot section and setting the level low so the ntp.conf is populated prior to ntpd starting on the firstboot (I use level 47 in the sample below).

Regards Paul

%firstboot --unsupported --interpreter=busybox --level=47

  1. SET ESXFIRSTBOOT1 VARIABLES

/bin/echo "Set esxfirstboot1 Variables"

NTPSERVERS="server1.x.x server2.x.x server3.x.x"

/bin/echo ""

  1. CONFIGURE NTP

/bin/echo "Configuring NTP"

/bin/echo "NTPSERVERS Variable = $" for NTPSERVER in $;

do

/bin/echo "server $" >> /etc/ntp.conf

done

/bin/echo "NTP Configuration"

/bin/cat /etc/ntp.conf

/bin/echo ""

0 Kudos
lamw
Community Manager
Community Manager

I actually tried and it did not work all the way. If you login to vSphere Client, you'll notice the NTP servers are not populated. In any case, you'll be going through a reboot after your patching, so there's really no need to change any other init scripts. The reboot will handle all conf changes for you the next time the system boots up.

=========================================================================

William Lam

VMware vExpert 2009,2010

VMware scripts and resources at:

Twitter: @lamw

Getting Started with the vMA (tips/tricks)

Getting Started with the vSphere SDK for Perl

VMware Code Central - Scripts/Sample code for Developers and Administrators

VMware Developer Community

If you find this information useful, please award points for "correct" or "helpful".

0 Kudos
SuperSpike
Contributor
Contributor

Can someone post an example ks.cfg script? I've tried creating the two %firstboot sections as described in this thread, but the patches are not getting downloaded or installed. I'd like to verify my setup against a known good config. Thanks.

@Virtual_EZ
0 Kudos