Print | Rate this content

Advisory: (Revision) PCI Devices May Be Enumerated in a Different Order with Red Hat Enterprise Linux 4 or 5 (or Later), or SUSE Linux Enterprise Server 9 (or Later) May Require Configuring Option PCI Devices When Added After the Installation

SUPPORT COMMUNICATION - CUSTOMER ADVISORY

Document ID: c01430330

Version: 2

Advisory: (Revision) PCI Devices May Be Enumerated in a Different Order with Red Hat Enterprise Linux 4 or 5 (or Later), or SUSE Linux Enterprise Server 9 (or Later) May Require Configuring Option PCI Devices When Added After the Installation
NOTICE: The information in this document, including products and software versions, is current as of the Release Date. This document is subject to change without notice.

Release Date: 2009-12-08

Last Updated: 2009-12-08


DESCRIPTION

Document Version
Release Date
Details
2
12/08/2009
Updated Description and Scope section to include SUSE Linux Enterprise Server 9 (or later).
1
04/11/2008
Original Document Release.

Beginning with 2.6.x kernels (Red Hat Enterprise Linux 4 or later and SUSE Linux Enterprise Server 9 or later), the default PCI device enumeration algorithm uses a depth-first algorithm instead of the breadth-first algorithm. This change causes PCI devices to be enumerated in a different order under the Linux kernel than reported by the Power-On Self Test (POST) or the ROM-Based Setup Utility (RBSU). For example, the Network Interface Controller (NIC) 1 in the RBSU may not be identified to the kernel as eth0 as expected. This will also occur with other PCI devices such as storage controllers.

After the operating system installation, if optional PCI devices are added to the server, and the PCI slots into which these optional PCI devices are inserted are in lower-numbered busses than embedded or previously installed devices of the same type (NIC, SCSI, etc), the enumeration of all such devices will be changed.

This change in enumeration could possibly render the server unbootable if the additional PCI option card is a storage controller and the boot device or root partition is no longer located at the designated controller specified by the boot loader.

Starting with 2.6.19-rc3 kernels, a patch was developed containing a whitelist of servers with PCI option slots that are in lower-numbered buses than the embedded devices. By default, on these kernels, the servers in this whitelist have the PCI enumeration algorithm set to breadth-first, rather than the default PCI default depth-first algorithm. This has the same effect as including the following as a kernel boot parameter during boot:

pci=bfsort

However, the whitelist patch is not the default when performing a new installation of Linux. This occurs because the anaconda installation kernel uses depth-first algorithm that creates configuration data files (such as "/etc/sysconfig/hwconf" and "/etc/sysconfig/network-scripts/ifcfg-eth*") with embedded device identification (e.g., MAC address) from the depth-first order.

Subsequent boots will enumerate the devices based on the configuration files regardless of the patch. This will continue even if the pci=bfsort kernel boot parameter is implemented.

When configuring the server, the differences between how the option PCI devices are enumerated needs to be understood so that the /etc/sysconfig/hwconf and the /etc/sysconfig/network-scripts/ifcfg-eth* files can be modified or deleted as necessary for devices to enumerate correctly after subsequent boots.

SCOPE

Any ProLiant server running Red Hat Enterprise Linux 4, Red Hat Enterprise Linux 5, SUSE Linux Enterprise Server 9, SUSE Linux Enterprise Server 10, or SUSE Linux Enterprise Server 11 when adding an optional PCI device.

This is not ProLiant server-specific.

RESOLUTION

To ensure that PCI devices are enumerated as expected by maintaining the breadth-first sort mechanism, perform the following:

  • In an interactive installation, add the "pci=bfsort" onto the Boot: line of initial installation screen.
  • For PXE boot, include "pci=bfsort" on the parameter list in the appropriate pxelinux.cfg file.
  • In kickstart files, include " --append="pci=bfsort" on the bootloader keyword directive to have the parameter automatically added to grub.conf after the installation is complete.

Other Considerations

Device names can change if, after installation, another Smart Array controller is added and depending on where it is inserted in the bus. For example, a configuration that has an embedded Smart Array controller and another Smart Array controller is added to an option PCI slot, such that /dev/cciss/c0d0 and /dev/cciss/c1d0 exists.

If another controller is added, even with the bfsort option set, and depending on where it shows up on the bus, what was /dev/cciss/c1d0 could become /dev/cciss/c2d0.

This can be circumvented if device "labels" instead of device names are used in /etc/fstab.

RECEIVE PROACTIVE UPDATES : Receive support alerts (such as Customer Advisories), as well as updates on drivers, software, firmware, and customer replaceable components, proactively via e-mail through HP Subscriber's Choice. Sign up for Subscriber's Choice at the following URL:


NAVIGATION TIP : For hints on navigating HP.com to locate the latest drivers, patches, and other support software downloads for ProLiant servers and Options, refer to the Navigation Tips document .
SEARCH TIP : For hints on locating similar documents on HP.com, refer to the Search Tips document .
To search for additional advisories related to Red Hat Enterprise Linux, use the following search string:
+Advisory +ProLiant -"Software and Drivers" +Red Hat Enterprise Linux

Hardware Platforms Affected: HP ProLiant BL280c G6 Server Blade, HP ProLiant BL460c G6 Server Blade, HP ProLiant BL465c G6 Server Blade, HP ProLiant BL490c G6 Server Blade, HP ProLiant BL495c G5 Server Blade, HP ProLiant BL495c G6 Server Blade, HP ProLiant BL680c G5 Server Blade, HP ProLiant BL685c G5 Server Blade, HP ProLiant BL685c G6 Server Blade, HP ProLiant DL120 G5 Server, HP ProLiant DL120 G6 Server, HP ProLiant DL140 G2 Server, HP ProLiant DL140 G3 Server, HP ProLiant DL145 G2 Server, HP ProLiant DL145 G3 Server, HP ProLiant DL160 G5 Server, HP ProLiant DL160 G5p Server, HP ProLiant DL160 G6 Server, HP ProLiant DL160se G6 Server, HP ProLiant DL165 G5 Server, HP ProLiant DL165 G5p Server, HP ProLiant DL165 G6 Server, HP ProLiant DL180 G5 Server, HP ProLiant DL180 G6 Server, HP ProLiant DL180 Server, HP ProLiant DL185 G5 Server, HP ProLiant DL320 G2 Server, HP ProLiant DL320 G3 Server, HP ProLiant DL320 G4 Server, HP ProLiant DL320 G5 Server, HP ProLiant DL320 G5p Server, HP ProLiant DL320 G6 Server, HP ProLiant DL320s Server, HP ProLiant DL360 G2 Server, HP ProLiant DL360 G3 Server, HP ProLiant DL360 G4 Server, HP ProLiant DL360 G4p Server, HP ProLiant DL360 G5 Server, HP ProLiant DL360 G6 Server, HP ProLiant DL365 G5 Server, HP ProLiant DL365 Server, HP ProLiant DL370 G6 Server, HP ProLiant DL380 G2 Server, HP ProLiant DL380 G3 Server, HP ProLiant DL380 G5 Server, HP ProLiant DL380 G6 Server, HP ProLiant DL385 G2 Server, HP ProLiant DL385 G5 Server, HP ProLiant DL385 G5p Server, HP ProLiant DL385 G6 Server, HP ProLiant DL385 Server, HP ProLiant DL560 Server, HP ProLiant DL580 G2 Server, HP ProLiant DL580 G3 Server, HP ProLiant DL580 G4 Server, HP ProLiant DL580 G5 Server, HP ProLiant DL585 G2 Server, HP ProLiant DL585 G5 Server, HP ProLiant DL585 G6 Server, HP ProLiant DL585 Server, HP ProLiant DL740 Server, HP ProLiant DL760 G2 Server, HP ProLiant DL760 Server, HP ProLiant ML110 G2 Server, HP ProLiant ML110 G3 Server, HP ProLiant ML110 G4 Server, HP ProLiant ML110 G5 Server, HP ProLiant ML110 G6 Server, HP ProLiant ML115 G5 Server, HP ProLiant ML115 Server, HP ProLiant ML150 G2 Server, HP ProLiant ML150 G3 Server, HP ProLiant ML150 G5 Server, HP ProLiant ML150 G6 Server, HP ProLiant ML310 G2 Server, HP ProLiant ML310 G3 Server, HP ProLiant ML310 G4 Server, HP ProLiant ML310 G5 Server, HP ProLiant ML310 G5p Server, HP ProLiant ML330 G2 Server, HP ProLiant ML330 G3 Server, HP ProLiant ML330 G6 Server, HP ProLiant ML330e Server, HP ProLiant ML350 G2 Server, HP ProLiant ML350 G3 Server, HP ProLiant ML350 G4 Server, HP ProLiant ML350 G4p Server, HP ProLiant ML350 G5 Server, HP ProLiant ML350 G6 Server, HP ProLiant ML370 G2 Server, HP ProLiant ML370 G3 Server, HP ProLiant ML370 G4 Server, HP ProLiant ML370 G5 Server, HP ProLiant ML370 G6 Server, HP ProLiant ML530 G2 Server, HP ProLiant ML570 G2 Server, HP ProLiant ML570 G3 Server, HP ProLiant ML570 G4 Server
Operating Systems Affected: Red Hat Enterprise Linux 4 (x86), Red Hat Enterprise Linux 4 (x86-64), Red Hat Enterprise Linux 5 (AMD64/EM64T) SW(Standard HP Product), Red Hat Enterprise Linux 5 (x86), SUSE Linux Enterprise Server 10 (AMD64/EM64T) SW(Standard HP Product), SUSE Linux Enterprise Server 10 (x86) SW(Standard HP Product), SUSE Linux Enterprise Server 11 (AMD64/EM64T)(Standard HP Product), SUSE Linux Enterprise Server 11 (x86), SUSE Linux Enterprise Server 9 (AMD64/EM64T)(Standard HP Product), SUSE Linux Enterprise Server 9 (x86)(Standard HP Product)
Software Affected: Not Applicable
Support Communication Cross Reference ID: IA01430330
©Copyright 2009 Hewlett-Packard Development Company, L.P.
Hewlett-Packard Company shall not be liable for technical or editorial errors or omissions contained herein. The information provided is provided "as is" without warranty of any kind. To the extent permitted by law, neither HP or its affiliates, subcontractors or suppliers will be liable for incidental,special or consequential damages including downtime cost; lost profits;damages relating to the procurement of substitute products or services; or damages for loss of data, or software restoration. The information in this document is subject to change without notice. Hewlett-Packard Company and the names of Hewlett-Packard products referenced herein are trademarks of Hewlett-Packard Company in the United States and other countries. Other product and company names mentioned herein may be trademarks of their respective owners.

Provide feedback

Please rate the information on this page to help us improve our content. Thank you!