Updating p5 System Firmware
May 2006
1 Overview
1.1 p5 server hardware
1.2 System firmware fixes and upgrades
1.3 Decoding firmware names
1.3.4 Why are there multiple firmware streams?
1.4 Decoding the operator control panel
1.5 Temporary versus Permanent Firmware sides
1.6 Be Advised
2 Requirements
2.1 Software requirements
3 Upgrade firmware with HMC
3.1 Connectivity
3.2 HMC requirements
3.3 Viewing system firmware information using LIC wizard
3.4 Upgrade system firmware to a new release using HMC
3.5 Update system firmware within a release using HMC
3.6 Change License Internal Code Wizard
3.7 Reject the installed firmware using a HMC
4 Upgrade firmware without a HMC
4.1 Access ASMI via serial console
4.2 Checking the current firmware level
4.3 Power on using ASMI
4.4 Upgrade system firmware via running operating system
4.4.1 Upgrade firmware image using AIX
4.4.2 Upgrade firmware image using Linux
4.5 Upgrade firmware using diagnostics CD
5 Reject installed firmware without an HMC
5.1 Boot to the permanent side
5.2 Reject the installed firmware using an OS command
5.3 Reject the installed firmware using a diagnostic CD
6 Appendix common problems
7 References
1 Overview
The goal of this paper is to provide easy-to-read instructions to quickly update system firmware on p5 servers. It is assumed the reader has basic p5 skills. There are extensive references that should help with items not covered in this paper.
Three firmware update methods will be covered:
1. Update on an HMC managed system
2. Update on a standalone server via OS, without a HMC
3. Update on a standalone server using the Diagnostic CD, without a HMC
Please first read through the prerequisites before getting started on any of these upgrade sections. Then refer to any one section or all sections when choosing the preferred method for updating system firmware.
1.1 p5 server hardware
The hardware used in developing this paper was a p5 550Q (type-model 9133-55A). HMC (version 5.2.0 including fix MH00586) was also used when needed.
1.2 System firmware fixes and upgrades
Firmware, also known as microcode, is Licensed Internal Code that fixes problems and enables new system features as they are introduced. New features introduced are supported by new firmware release levels. In between new hardware introductions, there are fixes or updates to the supported features. These fixes are often bundled into service packs. A service pack is referred to as an update level. A new release is referred to as an upgrade level. Both levels are represented by the file name in the form of PPMMXXX_YYY_ZZZ. PP and MM are package and machine type identifiers. PP can be 01 for managed system or it can be 02 for power subsystem. The MM identifier is a SF for p5 systems and a BP for Bulk Power Controller. The firmware version file applicable to p5 machines is in the form of 01SFXXX_YYY_ZZZ.
1.3 Decoding firmware names
The file naming convention for system firmware is:
01SFXXX_YYY_ZZZ, where
XXX is the stream release level
YYY is the service pack level
ZZZ is the last disruptive service pack level
Using the above example, the system firmware 01SF235_185 would be described as release level 235, service pack 185.
Each stream release level supports new machine types and/or new features.
Firmware updates can be disruptive or concurrent. A disruptive upgrade is defined as one that requires the target system to be shutdown and powered off prior to activating the new firmware level. A new release level upgrade will always be disruptive. All other upgrades are defined as concurrent, meaning that they can be applied while the system is running. Concurrent updates require an HMC but are not guaranteed to be non-disruptive.
In general, a firmware upgrade is disruptive if
1. The release levels (XXX) are different.
Example: Currently installed release is SF230, new release is SF235
2. The service pack level (YYY) and the last disruptive service pack level (ZZZ) are equal.
Example: SF235_180_180 is disruptive, no matter what level of SF235 is currently installed on the system
3. The service pack level (YYY) currently installed on the system is lower than the last disruptive service pack level (ZZZ) of the new service pack to be installed.
Example: Currently installed service pack is SF235_180_180 and the new service pack is SF235_190_185
An installation is concurrent if:
1. The service pack level (YYY) is higher than the service pack level currently installed on your system.
Example: Currently installed service pack is SF235_180_160, new service pack is SF235_185_160.
1.3.4 Why are there multiple firmware streams?
Multiple firmware streams (eg, SF230, SF235, SF240, etc) are available for a given type-model (eg, 9117-570). IBM maintains multiple parallel firmware streams so customers can install firmware fixes while avoiding a scheduled p5 server outage. As described on the Managing and installing system firmware FAQ web page, IBM releases parallel firmware streams (release levels) which are very analogous to the AIX V5.2 and V5.3 release levels. Like AIX V5.2 and V5.3, the firmware streams perform similar functions on the same machine type-models, but SF240 (for example) may have some functions which SF235 (for example) does not. There is usually little or no reason to upgrade from one release to another (eg, SF235 to SF240), but if the upgrade is performed, it requires an outage of the entire p5 server (every LPAR at once). In contrast, for systems managed by an HMC, updating from one service pack level to another (within the same stream release level) is often not disruptive.
1.4 Decoding the operator control panel
When the system is powered on, note the operator control panel. It should appear similar to the image below.
In this example the system is currently booted from the temporary side of the firmware image as denoted in the control panel by the letter T. This indicates the firmware is running from the temporary side. N indicates the system is booted in normal mode. V=F indicates the boot speed is set to Fast. HMC=1 indicates that the server is managed by and connected to one HMC.
If it has been recently managed by an HMC and no HMC is connected then it will display HMC=0. If no HMC is available and it is desired to set the server to unmanaged it might be required to reset the service processor to factory default using ASMI.
1.5 Temporary versus Permanent Firmware sides
The Service Processor maintains two copies of firmware, the temporary and permanent side, to help manage and reduce the frequency of downtime for maintenance. The permanent side is also known as the "P" side. The temporary side is also known as the "T" side. Server firmware fixes are installed on the temporary side. Copying the temporary firmware level to the permanent side is known as committing or accepting the fix. Conversely, rejecting, or removing the current firmware level consists of copying the permanent firmware image to the temporary side.
Note: It is recommended to use a new firmware fix for a period of time before committing (or accepting) it.
If firmware fixes are applied consecutively, the first fix will, by default, be copied from the temporary to the permanent side, or accepted. Using an HMC, it is possible to simply replace the temporary image by doing an Install and Activate of the new firmware and indicating that the firmware should not be accepted.
1.6 Be Advised
During a firmware update, the flashing of the NVRAM might take anywhere from ten minutes to one hour. In general, updating to a new release level will take longer. Ensure the system is not interrupted before the flash process is completed. Interrupting this process could result in a service call.
For systems that are not managed by a HMC, the installation of system firmware is always disruptive. During the update_flash process, the console output will be displayed. Again, do not interrupt this process.
2 Requirements
2.1 Software requirements
The table below is a summary of the minimum components required for each method covered in this paper:
Method | Minimum Requirements |
Update via HMC | 1. A compatible version of HMC. 2. An Ethernet connection from the HMC to the p5 server (HMC1 port). 3. Desired firmware image on CD. The rpm and XML files are required. |
Update via running AIX or Linux operating system | 1. A running AIX or supported Linux operating system on a single LPAR environment, ie, no attached HMC. 2. Firmware image on a CD or file system. The rpm file only. 3. update_flash executable. For AIX, it is part of the diagnostic aids tool in the /usr/lpp/diagnostics/bin directory. For Linux, it is part of the Service and Productivity Tools. 4. Serial console and connection |
Update via Standalone Diagnostic CD | 1. Diagnostic CD 2. Firmware image (.img) file on a CD. Remember, the rpm file is not directly compatible. 3. Serial console and connection |
There are two very helpful sites that will assist in gathering the components necessary to update firmware. Visiting the Microcode downloads site ( 1) is recommended before performing any updates.
http://www14.software.ibm.com/webapp/set2/firmware/gjsn
Figure 1
To download the rpm and XML files, input the server machine type and model number and select the latest firmware components based on the below requirements table (option 1 in Fig 3).
If planning to using the Diagnostic CD method, use option 4 to download the firmware image ISO image. The rpm files are not directly compatible with the Diagnostics CD. For the smallest ISO image take the following path using option 4
Next, go to the Power5 Code matrix site (Figure 2) to ensure the existing code levels support the downloaded firmware release. For the purposes of this paper, this applies mostly to HMC version level. If an HMC is not being used, this is for information only.
https://www14.software.ibm.com/webapp/set2/sas/f/power5cm/home.html
Figure 2
3 Upgrade firmware with HMC
3.1 Connectivity
The HMC is connected to a p5 system with a standard TCP/IP Ethernet connection. Figure 1 shows a very simple connection.
Figure 3.
p550Q server with a direct connection to a HMC.
3.2 HMC requirements
Before applying the latest level of firmware on the system using a HMC it might be required to update the build version. Please see the recommended HMC level for the target firmware level in the Power5 Code Matrix. It is always a good idea to have a suitable backup of the HMC data before updating HMC code.
In this example, per the Power5 Code Matrix for system firmware SF240_202, the recommended code level for our HMC is Version 5.2 plus service pack MH00586. This is also the most current level at the writing of this paper.
Check the HMC version build level by clicking on the HMC Code Update in the HMC navigation area (Figure 4):
Figure 4
The HMC version can also be checked as hscroot from the shell prompt as follows:
3.3 Viewing system firmware information using LIC wizard
From the navigation area, select License Internal Code Updates. In the content area, click on Change Licensed Internal Code to the current release. In the Target Object Selection window, click the target system, and click OK.
Figure 5
In the Change Internal Code window, select View system information and click OK. From the main Change Internal Code task panel, select View system information and click OK. To view the installed, activated, and accepted LIC levels on the target, select "None" on the Specify LIC Repository panel and click OK.
Figure 6
3.4 Upgrade system firmware to a new release using HMC
Remember, upgrading to a new release is a disruptive upgrade.
Start by opening the Server and Partition folder in the HMC. Then, click on Server Management in that folder. If the state of the machine is Power off, Ready, or Standby, then proceed. Setting the state to Power off is recommended when performing a firmware upgrade, although it is not required. Note: only HMC managed systems can perform firmware upgrades with target system set to Power off state.
Figure 7
Next, open the Licensed Internal Code Maintenance folder on the Hardware Management Console. Then, click on Licensed Internal Code Updates in that folder. In this example, the update will be from our current firmware level 01SF235_185 to 01SF240_202, so the normal " Change Licensed Internal Code for the current release" feature will not work. Select Upgrade Licensed Internal Code to a new release.
Figure 8
Select the desired target managed system and click OK.
Figure 9
Insert the CD with the rpm and XML files into the drive. On the Specify LIC Repository panel, select DVD drive and click OK.
Next, a Select LIC level panel is shown. Click OK.
Figure 11
The next prompt will be to accept the LIC license agreement for machine code. Read the license and click OK to accept. After accepting the license agreement, confirm the disruptive upgrade action. Click OK to proceed. When the firmware is flashed, the FSP will restart and activate the new firmware level.
Figure 12
A dialog box will appear to showing the elapsed time and status of the firmware upgrade.
WARNING - During a disruptive update, the flashing of NVRAM might take from ten minutes to two hours. Do not interrupt the process before the flash process is complete.
Figure 13
Once the firmware upgrade has completed, view the system firmware information to see how the upgrade has changed what is available.
3.5 Update system firmware within a release using HMC
Firmware updates within a release are common when maintaining and managing a p5 server.
Use the Change Licensed Internal Code for the current release feature to install LIC updates on p5 servers. Updates can be applied to managed systems, bulk power controllers, and I/O adapters. This example shows updating a managed systems LIC from an installed firmware level of 01SF235_180_160 to 01SF235_185_160. Note that the release level will remain the same, but the service pack level will be updated from 180_160 to 185_160. The update is concurrent, or non-disruptive, as indicated in the service pack (185) and the last disruptive (160) levels.
3.6 Change License Internal Code Wizard
Start by opening the Server and Partition folder in the HMC. Then, click on Server Management. Verify that the target system exists in the content area.
Figure 14
From the HMC left navigation area, open the Licensed Internal Code Maintenance folder. Click on Licensed Internal Code Updates. Select the Change Licensed Internal Code for the current release feature.
Highlight the target managed system and click OK (Figure 15).
Figure 15
Click the OK button to start the Change Licensed Internal Code wizard.
Figure 16
Insert the CD into the drive and on the Specify LIC Repository panel, select DVD drive and click OK.
The following panel is displayed providing summary information about the update. Note the Advanced Options button gives the option to view and change the installation type for this update action. The current install type selection is Concurrent install and activate. Other installation types involve either a deferred or an immediate system restart to activate the new firmware level. Those installation types are grayed out since this is a concurrent update only. (See Managed System and Power Licensed Internal Code (LIC) Concurrency panel below.)
Click Next to continue with Concurrent install and activate. Please read the license agreement and click OK to accept.
Figure 16
The Advanced Option button displays the Managed System and Power Licensed Internal Code (LIC) Concurrency panel:
Figure 17
After accepting the license agreement, confirm the concurrent update action prompt. Click Finish and Close to proceed.
Figure 18
A progress window will appear to show the elapsed time and status of the firmware update. When the firmware is flashed, the FSP will restart and activate the new firmware level.
Figure 19
Once the firmware upgrade has completed, view the system firmware information to see how the upgrade has changed what is available.
3.7 Reject the installed firmware using a HMC
From the HMC left navigation area, select License Internal Code Updates.
In the content area, click Change Licensed Internal Code to the current release. In the Target Object Selection window, click the target system, and click OK.
The main panel then displays with three options start the Change Internal Code wizard, view system information, and select advanced features.
Figure 20
The Remove and activate feature is a one step process to remove the current active T-side firmware and roll back (or copy) firmware from the P-side. Essentially, undo the last firmware update and restore the T-side with the P-side firmware version.
Figure 21
4 Upgrade firmware without a HMC
4.1 Access ASMI via serial console
A system with no HMC is also known as an unmanaged system. ASMI is used to power on the system and perform other useful functions. Using a serial cable and a program like HyperTerminal on Windows, ASMI and the active console can be accessed. Other communication programs should work.
Figure 22.
Back view of p550Q server with ports for (left to right)
serial, SPCN, HMC, USB and Ethernet.
Once the serial connection to the system is established, press the key, to be presented with the following ASMI login screen.
Type the User ID and password to log in to ASMI. If this is the first time logging into ASMI, it might be required to change the default password. The default password is admin
4.2 Checking the current firmware level
Upon logging into ASMI, the firmware level will be clearly displayed as shown below.
The current activated firmware level is also shown in the ASMI web interface after logging in. To see the firmware level, look in the upper-right corner. Basically, to access the ASMI web interface, connect an Ethernet cable directly to the HMC1 port. For details on accessing the ASMI web interface, see reference 6.
Figure 23
The current active firmware level can also seen from the output of the Display Microcode Level selection on the Diagnostics CD. Details of how to get to the following screen are provided in 4.7
Figure 24
The current active firmware level as seen from the SMS screen:
Figure 25
Finally, if the server has a running AIX or Linux operating system with the Service and Productivity tool lsvpd installed, the lsmcode command can be used as shown below. More details are provided in section 4.4.2
4.3 Power on using ASMI
From the ASMI main menu select 1. Power/Restart Control to get to this screen:
Select 1. Power On/Off System.
On the next screen select 8. Power on. Wait for a few seconds to be logged out as the system powers on. Watch boot progress codes as the system comes up.
When the system completes the boot process, note the operator control panel. Note that there should be no indication of HMC=. This indicates that the service processor does not expect to be managed by an HMC (see below).
4.4 Upgrade system firmware via running operating system
The rpm file for the firmware fix file stored either in the file system or on a mounted CD. For this example, the rpm file is in the /tmp directory.
Run the command below to extract the flash image file in the rpm file:
rpm -Uvh --ignoreos /tmp/01SF240_202_201.rpm
The flash image file is put into a newly created directory /tmp/fwupdate
To install the server firmware through a running OS, use the update_flash. To run this command, root authority is required. Since installing server firmware fixes through the operating system is a disruptive process, shut down any running applications and logout any non-root users.
On AIX, the update_flash command is located in the /usr/lpp/diagnostics/bin directory. If this directory does not exist, install the AIX diagnostics to run this command.
On Linux, the update_flash command is located in the /usr/sbin directory. A separate installation of Service and Productivity Tools may be required.
The command syntax is as follows (for both AIX and Linux):
4.4.1 Upgrade firmware image using AIX
Before installing check the existing firmware level. From AIX, use the command lsmcode. This command resides in the diagnostic directory. An example of the output of the lsmcode command is as follows:
Next, run the update_flash command to upgrade firmware:
4.4.2 Upgrade firmware image using Linux
To use lsmcode and update_flash commands in Linux, the following minimum Service and Productivity tools have to be installed, and in this order:
librtas
ppc64-utils
lsvpd
Note: If the Service and Productivity tools are not installed, they can be downloaded from the following location. Choose the appropriate Linux distribution and release.
https://www14.software.ibm.com/webapp/set2/sas/f/lopdiags/home.html
After downloading the Linux tools into the /tmp directory, install the three packages - Platform Enablement Library (librtas), Service Aids (ppc64-utils), and Hardware Inventory (lsvpd) as follows
The Hardware Inventory package contains the lsmcode command that allows us to view the current system firmware information. To populate the VPD database and enable this command, type
Run the update_flash command to upgrade firmware:
The flash process will reboot the system as part of a disruptive install of the firmware. Do Not Power off or interrupt the flash process. When the new firmware is completely installed, log back into Linux and verify the current firmware configuration.
4.5 Upgrade firmware using diagnostics CD
This method is intended for systems that are not managed by a HMC.
Insert the diagnostics CD into the CD-ROM drive and connect to ASMI to power on or restart the system as described in 4.3.
Press 1 after keyboard POST to in order to get the SMS menu screen. Watch closely as this passes quickly.
Select the following SMS options and choose to boot from the CD device.
This will look like a normal AIX boot with Welcome to AIX scrolling across the console.
Once the Diagnostics disk has booted, on the function selection screen, select Tasks and Service Aids.
Figure 27
On the task selection screen, scroll to the bottom of the list of options, and select Update and Manage Flash.
Figure 28
Insert the CD containing the downloaded firmware img file into the media device, select Validate and Update System Firmware and Update System Firmware menu items.
Figure 29
Press F4 to select a firmware fix image file from a list of image files. Our CD contains only one version of the target firmware update file, and 01SF204_202_201.img is displayed.
Figure 30
Press F7 (commit). The server firmware level selected will be installed on the temporary side.
5 Reject installed firmware without an HMC
There are times when it may be necessary to reject a firmware update. The reject function is accomplished by basically copying the P-side firmware to the T-side and activating. The rejected T-side firmware is removed completely. Without an HMC, it is possible to reject the firmware in the T-side and roll back to the firmware in the P-side using the OS update_flash command or a Diagnostics CD.
Note: If rejecting firmware without an HMC, the server must be booted from the P-side copy of the firmware prior to performing this action.
5.1 Boot to the permanent side
Connect to ASMI as described in section 4.1. Log in to ASMI and select 1. Power/Restart Control
Select 1. Power On/Off System
Select 2. boot side for the next boot
Select 1. Permanent
Note that the firmware boot side is now set to Permanent which is our backup copy of the firmware flash.
Select 8. Power on. Hit the key. Wait for a few seconds to be logged out as the system powers on.
5.2 Reject the installed firmware using an OS command
To reject firmware using AIX or Linux, use the update_flash command with the r option. The system is running on the P-side firmware as noted in the lsmcode output SF235_185 (b). (b) denotes booted.
This option will reject the installed T-side firmware, SF240_202 (t), and copy SF235_185 (p) to the T-side copy of the firmware. The system will reboot.
5.3 Reject the installed firmware using a diagnostic CD
Powered on from the P-side (see 5.1), boot the Diagnostics CD as described in section 4.5. Then select Update and Manage Flash selection from the diagnostics CDs Task Selection List. Note: the system is currently booted from the permanent firmware side.
Move the highlighted cursor to Reject the Temporary Image and hit key to proceed with rejecting the installed firmware on the T-side.
6 Appendix common problems
1. Using HMC LIC wizard, result is error code HSCP0155 The task is unavailable. Please try again later.
Please refer to 2.3 HMC requirements and ensure the HMC build level is up-to-date. [7]
2. A message stating: "This partition does not have the authority to perform the requested function. is received.
Before doing firmware update other than using HMC LIC feature, check the Firmware Update Policy in the ASMI. For the HMC LIC method, the policy should be set to HMC. For other methods, the policy should be set to Operating System.
3. Where do I get the DIAG CD and other required components
See [14] in 7 References for download sites.
4. Why is the firmware in rpm and xml formatted files? Is rpm command in the AIX toolbox as a separate installation? Isnt rpm an alien format in AIX?
The rpm in AIX toolbox has been available on AIX and has been shipped with AIX since v5.2. AIX geninstall command uses the rpm command for installing rpm filesets.
5. Received error: Failed dependencies: messages during installation of Linux Service and Productivity Tools
Use the rpm option requires to look for the missing prerequisite rpm package. For instance, the dependency file, librtas rpm, is required by the ppc64-utils package
7 References
[1] Firmware update maintenance documentation
http://publib.boulder.ibm.com/infocenter/eserver/v1r3s/topic/ipha5/fixes.htm
[2] InfoCenter documentation
http://publib.boulder.ibm.com/eserver/
[3] Hardware Management Console for p5 Operations Guide - how to use an IBM Hardware Management Console for p5 (HMC) to manage a system.
http://publib16.boulder.ibm.com/pseries/en_US/infocenter/base/hardware_docs/pdf/380590.pdf
[4] Hardware Management Console corrective service support
https://www14.software.ibm.com/webapp/set2/sas/f/hmc/home.html
[5] IBM p5 hardware documentation
http://publib16.boulder.ibm.com/pseries/en_US/infocenter/base/hardware.htm
[6] Accessing the ASMI Interface
http://publib.boulder.ibm.com/infocenter/eserver/v1r3s/topic/iphby/iphby.pdf
[7] Recommended p5 firmware levels
https://www14.software.ibm.com/webapp/set2/sas/f/power5cm/home.html
[8] Firmware downloads
http://www14.software.ibm.com/webapp/set2/firmware/
[9] Firmware for 9133-55A
http://www14.software.ibm.com/webapp/set2/firmware/gjsn?mode=1&mtm=9133-55A
[10] System Firmware SF235_185
http://www14.software.ibm.com/webapp/set2/firmware/gjsn?mode=30&page=01SF235_185_160.html
[11] System Firmware SF240_202
http://techsupport.services.ibm.com/server/mdownload2/01SF240_202_201.html
[12] Update firmware via HMC
http://publib.boulder.ibm.com/infocenter/eserver/v1r3s/topic/ipha5/getspecificfix.htm
[13] Instructions for installing firmware updates and upgrades can be found at
http://publib.boulder.ibm.com/infocenter/eserver/v1r3s/index.jsp?topic=/ipha5/fix_serv_firm_kick.htm
[14] Download standalone Diagnostics CD-ROM
https://www14.software.ibm.com/webapp/set2/sas/f/diags/home.html
[15] Booting Diagnostics CD
http://publib.boulder.ibm.com/infocenter/eserver/v1r3s/topic/ipha5/standalone_procedure_nohmc.htm
[16] Service Aids for Linux (Service toolkit)
https://www14.software.ibm.com/webapp/set2/sas/f/lopdiags/home.html
[17] Service and productivity tools for Linux
http://publib.boulder.ibm.com/infocenter/eserver/v1r3s/index.jsp?topic=/ipha5/obtain_service_tools.htm
[18] p5 Reference Codes
http://publib.boulder.ibm.com/infocenter/eserver/v1r3s/topic/ipha6/ipha6.pdf
[19] AIX Tasks and Service Aids
http://publib.boulder.ibm.com/infocenter/eserver/v1r3s/index.jsp?topic=/iphau/aixserviceaids.htm
[20] Resetting the server to a nonpartitioned configuration
http://publib.boulder.ibm.com/infocenter/eserver/v1r2s/en_US/index.htm?info/iphbl/iphblresetserver.htm
[21] Quick links for AIX fixes
http://www.ibm.com/servers/eserver/support/unixservers/aixfixes.html
[22] Support for AIX 5L and Linux servers
http://www.ibm.com/servers/eserver/support/unixservers/index.html