Archive for the ‘ESXi 4.1 Update 2’ Tag

Loss of network interfaces after applying CentOS kernel 2.6.18-308.4.1 on ESXi 4.1 Update 2

One of our hosting company requires us to keep up to date with RHEL 5 kernels on a server they support. As this is a production machine it means that when a new kernel is available I start applying it to development machines and migrate the upgrade through to production over a period of 3-4 weeks.

 

The first stage of testing is to apply the kernel on various ESXi hosted VMs and a couple of real servers, most of which run CentOS 5. The most recent kernel 2.6.18-308.4.1 seems to work OK, but on all of the ESXi hosts there seems to be an issue with the virtual network interfaces. In all cases I use the VMXNET 3 adapter type. The hosts are running ESXi 4.1 Update 2 with a number of HP provided bundles.

 

After the application of the updates and the subsequent reboot, only the local loopback network interface starts up. I’ve seen various suggestions as to what may be the cause but I can’t comment on those. To fix the issue, my initial test was to re-install the VMWare Tools which seems to do the trick. Having got that far, on a different guest I tried just rerunning vmware-config-tools.pl. That seems to solve the problem immediately (just restart the network service). The fix appears to be persistent across a reboot.

Applying ESXi 4.1 Update 2 to HP Proliant servers

I’m in the process of applying VMWare ESXi 4.1 Update 2 to some HP Proliant servers. Once I’d found the various steps required, the process was pretty straightforward. However, as is often the case when you’re on your own with a job you’ve not done before, the research/head scratching/gnashing of teeth took a lot longer than I’d expected. As it seems very poorly documented, I thought I would start this blog by detailing my experiences.

I only have a small number of servers to update. They were already running the HP OEM version of ESXi 4.1 Update 1. Some of the servers are local to me, the others are in a remote data centre. One of the local servers (a DL380 G5) is a sandpit machine that runs no meaningful loads. It is available to take the strain in case one of the main servers fails. The rest of the time it is used for testing. On all the servers, the disks containing guests are internal and are configured as raid 1 pairs. ESXi is installed on the first disk pair.

The environment I work in does not allow for guests to use vMotion. I can manually move guests and I had done so by moving a range of guests onto the test server (after it had had Update 2 applied) to check that the guests worked as expected and that VMWare tools updated without issue. These guests were then just moved back to their normal hosts after those hosts had been updated.

The HP OEM version of ESXi comes with various HP agents added in for things like hardware monitoring. If you just install the vanilla ESXi 4.1 you miss out on things that should make your life a lot easier when a power supply or fan fails. The original installation of ESXi 4.1 with Update 1 (with the HP bundle included) had been a simple matter of downloading the installation image from the VMWare web site. However, getting the update was more of an issue. VMWare only supplied the Update 2 bundle and made no mention of OEM updates. HP will supply a fresh installation image of ESXi 4.1 with Update 2 included but made no mention of a single image that would install both the VMWare and HP bundles combined.

So, the solution for me was to get the update bundles separately from VMWare and HP and apply both bundles to each server.

Starting from http://www.vmware.com/patchmgr/findPatch.portal I managed to download update-from-esxi4.1-4.1_update02.zip which gave me the ESXi 4.1 update 2. Finding the HP bundle proved much harder. So much so that in fact at one stage I assumed one didn’t exist. I eventually came across it in an HP Alert. The file I needed was called hp-esxi4.1uX-bundle-1.2-25.zip. Once you know what the file is called, it’s easy to find!

The bundles are applied by using the vihostupdate command. The server level commands were run from a vSphere Client via vCenter Server. The vihostupdate command is part of the vCLI, which in my case is installed on a separate, real server running RHEL 6.2 64bit. I applied the VMWare bundle, rebooted, then applied the HP bundle and rebooted. You can probably do both software updates with a single reboot. Having got the process to work with two reboots on my test server I stuck with that for the other servers. So, the process is as follows:

  • Shutdown the guests.
  • Put the host into maintenance mode.
  • Stop any guests from starting automatically when the host boots.
  • List what’s in the bundle
# vihostupdate --username=root --server=<ESXiHost> -l -b update-from-esxi4.1-4.1_update02.zip

Enter password:

---------Bulletin ID---------   ----------------Summary-----------------
ESXi410-201110201-SG            Updates ESXi 4.1 Firmware
ESXi410-201110202-UG            Updates ESXi 4.1 VMware Tools
ESXi410-Update02                VMware ESXi 4.1 Complete Update 2
  • List what’s already installed
# vihostupdate --username=root --server=<ESXiHost> -q
Enter password:
---------Bulletin ID--------- -----Installed----- ----------------Summary-----------------
ESXi410-201101223-UG          2011-01-13T05:09:39 3w-9xxx: scsi driver for VMware ESXi
ESXi410-201101224-UG          2011-01-13T05:09:39 vxge: net driver for VMware ESXi
hpq-esxi4.1uX-bundle-1.1      2011-03-31T11:26:48 HP ESXi 4.1 Bundle 1.1
hp-nmi-driver-1.2.02          2011-03-31T11:27:30 HP NMI Sourcing Driver for VMware ESX/ESXi 4.1
  • Apply the update “ESXi410-Update02”. Note that I mix my command switch formats a bit here, using -b and –bulletin in preference to using -b and -B.
# vihostupdate --username=root --server=<ESXiHost> --install -b update-from-esxi4.1-4.1_update02.zip 
  --bulletin ESXi410-Update02
Enter password:
Please wait patch installation is in progress ...
The update completed successfully, but the system needs to be rebooted for the changes to be effective.

When you run the above command, patience is a virtue. It takes a good couple of minutes during which time you get no progress information.

  • Now you can check to see that the installation worked by seeing what’s installed
# vihostupdate --username=root --server=<ESXiHost> -q
Enter password:
---------Bulletin ID--------- -----Installed----- ----------------Summary-----------------
hpq-esxi4.1uX-bundle-1.1      2011-03-31T11:26:48 HP ESXi 4.1 Bundle 1.1
hp-nmi-driver-1.2.02          2011-03-31T11:27:30 HP NMI Sourcing Driver for VMware ESX/ESXi 4.1
ESXi410-Update02              2012-03-02T13:56:58 VMware ESXi 4.1 Complete Update 2

We can see our ESXi410-Update02 bulletin is now reported.

  • As I mentioned above, at this point I rebooted the host but I guess you can just carry on and install the HP offline bundle.
  • Again, check what’s in the bundle
# vihostupdate --username=root --server=<ESXiHost> --list -b hp-esxi4.1uX-bundle-1.2-25.zip
Enter password:

---------Bulletin ID---------   ----------------Summary-----------------
hpq-esxi4.1uX-bundle-1.2-25     HP ESXi 4.1 Bundle 1.2-25
  • And apply it
# vihostupdate --username=root --server=<ESXiHost> --install -b hp-esxi4.1uX-bundle-1.2-25.zip 
  --bulletin hpq-esxi4.1uX-bundle-1.2-25
Enter password:
Please wait patch installation is in progress ...
The update completed successfully, but the system needs to be rebooted for the changes to be effective.

Once again, have a little patience whilst the installation takes place.

  • That’s the installation done, so now you have reboot the host. It should come back up in maintenance mode.
  • Take host out of maintenance mode and you’re ready to start booting the guests.
  • At some point, resume restarting guests when the host boots if that’s applicable in your environment.
  • Each guest will need VMWare tools updating and configuring – which requires the guests to be rebooted.