in both memory cells 0 and 1. A 24 DIMM memory carrier provides two 12-DIMM memory boards that hold four, eight, or twelve DIMMs in both memory cells 0 and 1.

All three versions of memory expanders must have their memory DIMMs installed in groups of four, known as a quad. DIMM quads of different sizes can be installed in any physical rank on all versions of memory expanders, but they must be grouped by their size.

Both the 24 and 48 slot memory expanders support physical memory ranks with four DIMMs, while the common 8 slot memory expander’s memory cells 0 and 1 each support physical ranks with two DIMMs. In the 8 slot memory expander, however, the logical quad of four DIMMs includes ranks from both sides 0 and 1 running in lock step with each other.

Memory DIMM Load Order

For a minimally loaded server, four equal-size memory DIMMs must be installed in slots 0A, 0B, 0C, and 0D on the same side of the 24/48 slot memory expander; and in the 0A and 0B slots on both 0 and 1 sides of the 8 slot memory expander.

The first quad of DIMMs are always loaded into rank 0’s slots for side 0 then in the rank 0’s slots for side 1. The next quad of DIMMs are loaded into rank 1’s slots for side 0, then for side 1, and so on, until all ranks slots for both sides are full.

Best memory subsystem performance result when both memory sides 0 and 1 have the same number of DIMM quads in them.

Memory Subsystem Behaviors

The zx2 chip in rx3600 servers provides increased reliability of memory DIMMs and memory carriers.

The zx2 chip doubles memory carrier error correction from 4 bytes to 8 bytes of a 128 byte cache line during cache line misses initiated by processor cache controllers, and by Direct Memory Access (DMA) operations initiated by I/O devices. This feature is called double DRAM sparing. 2 out of 72 DRAMs in any DIMM quad can fail without any loss of server performance.

You must replace DIMMs or memory carriers when a threshold is reached for multiple double-byte errors from one or more DIMMs on the same board. When any uncorrectable memory error (more than 2 bytes) or when no quad of like memory DIMMs is loaded in rank 0 of side 0, you must replace the DIMMs. All other DIMM errors are corrected by zx2 and reported to the Page Deallocation Table (PDT) and the diagnostic LED panel.

Memory Error Messages

•Diagnostic LEDs light only when an error is isolated to a specific DIMM.

•Configuration errors, such as no DIMMs installed, cause diagnostic LEDs to light for all DIMMs not installed.

•No diagnostic LEDs light for single-byte errors that are corrected in both Zx2 caches and memory DIMMs during corrected platform error (CPE) events. Diagnostic messages are reported for CPE events when thresholds are exceeded for both single-byte and double byte errors; all fatal memory errors cause global MCA events.

•PDT logs for all double byte errors are permanent. Single byte errors are initially logged as transient errors. If the server logs two single byte errors within 24 hours, they are upgraded to permanent in the PDT.

Table 5-17and Table 5-18list the memory subsystem events that light and that may light the diagnostic panel LEDs.

CPU, Memory and SBA 161