IBM 990 manual Book design

Draft Document for Review April 7, 2004 6:15 pm 6947ch02.fm

The following sections describe the z990 system structure, showing a logical representation of the data flow from PUs, L2 cache, memory cards, and MBAs, which connect I/O through Self-Timed Interconnect (STI).

2.2.2 Book design

A book has 12 PUs, two memory cards and three MBAs connected by the System Controller (SC). Each memory card has a capacity of 8 GB, 16 GB or 32 GB, resulting in up to 64 GB of memory Level 3 (L3) per book. A four-book z990 can have up to 256 GB memory. The Storage Controller, shown as SCC CNTLR in Figure 2-12 on page 40, acts as a cross-point switch between processor units (PUs), Memory Controllers (MSCs), and Memory Bus Adapters (MBAs).

The SD chips, shown as SCD in Figure 2-12,also incorporate a Memory Coherent Controller (MCC) function.

Each PU chip has its own 512 KB Cache Level 1 (L1), split into 256 KB for data and 256 KB for instructions. The L1 cache is designed as a store-through cache, meaning that altered data is also stored to the next level of memory (L2 cache). The z990 models A08, B16, C24, and D32 use the CMOS 9SG PU chips running at 0.83 ns.

The MCC controls a large 32MB L2 cache, and is responsible for the interbook communication in a ring topology connecting up to four books through two concentric loops, called the ring structure. The MCC optimizes cache traffic and will not look for cache hits in other books when it knows that all resources of a given logical partition are available in the same book.

The L2 cache is the aggregate of all cache space on the SD chips, resulting in a 32 MB L2 cache per book. The SC chip (SCC) controls the access and storing of data in the four SD chips. The L2 cache is shared by all PUs within a book and shared across books through the ring topology, providing the communication between L2 caches across books in systems with more than one book installed; the L2 has a store-in buffer design.

The interface between the L2 cache and processor memory (L3) is accomplished by four high-speed memory buses and controlled by the memory controllers (MSC). Storage access is interleaved between the storage cards, which tends to equalize storage activity across the cards. Each memory card has two ports that each have a maximum bandwidth of 8 GB/sec. Each port contains a control and a data bus, in order to further reduce any contention by separating the address and command from the data bus.

The memory cards support store protect key caches to match the key access bandwidth with that of the memory bandwidth.

The logical book structure is shown in Figure 2-12.

Chapter 2. System structure and design 39