Mtrr Mapping and Write Combining

A – Installation Troubleshooting BIOS Settings

MTRR is used by the InfiniPath driver to enable write combining to the QLogic on-chip transmit buffers. Write combining improves write bandwidth to the QLogic chip by writing multiple words in a single bus transaction (typically 64 bytes). Write combining applies only to x86_64 systems. To see if write combining is working correctly and to check the bandwidth, run the following command:

$ ipath_pkt_test -B

With write combining enabled, the QLE7140 and QLE7240 report in the range of 1150–1500 MBps; the QLE7280 reports in the range of 1950–2960 MBps. The QHT7040/7140 adapters normally report in the range of 2300–2650 MBps.

You can also use ipath_checkout (use option 5) to check bandwidth.

In some cases, the InfiniPath driver cannot configure the CPU write combining attributes for QLogic InfiniPath. This case is normally seen with a new system, or after the system’s BIOS has been upgraded or reconfigured.

If this error occurs, the interconnect operates, but in a degraded performance mode. The latency typically increases to several microseconds, and the bandwidth may decrease to as little as 200 MBps.

Upon driver startup, you may see these errors:

ib_ipath 0000:04:01.0: infinipath0: Performance problem: bandwidth to PIO buffers is only 273 MiB/sec

infinipath: mtrr_add(feb00000,0x100000,WC,0) failed (-22)

infinipath: probe of 0000:04:01.0 failed with error -22

If you do not see any of these messages on your console, but suspect this problem, check the /var/log/messages file. Some systems suppress driver load messages but still output them to the log file.

Two suggestions for fixing this problem are described in “Edit BIOS Settings to Fix MTRR Issues” on page A-2and “Use the ipath_mtrr Script to Fix MTRR Issues” on page A-3.

See the Troubleshooting section of the QLogic HCA and InfiniPath Software User Guide for more details on a related performance issue.

Edit BIOS Settings to Fix MTRR Issues

You can edit the BIOS setting for MTRR Mapping. The BIOS setting looks similar to:

MTRR Mapping

[Discrete]

For systems with very large amounts of memory (32GB or more), it may also be necessary to adjust the BIOS setting for the PCI hole granularity to 2GB. This setting allows the memory to be mapped with fewer MTRRs, so that there will be one or more unused MTRRs for the InfiniPath driver.

A-2	IB0056101-00 G