Chapter 2. System structure and design 39
Draft Document for Review April 7, 2004 6:15 pm 6947ch02.fm
The following sections describe the z990 system structure, showing a logical representation
of the data flow from PUs, L2 cache, memory cards, and MBAs, which connect I/O through
Self-Timed Interconnect (STI).
2.2.2 Book design
A book has 12 PUs, two memory cards and three MBAs connected by the System Controller
(SC). Each memory card has a capacity of 8 GB, 16 GB or 32 GB, resulting in up to 64GB of
memory Level 3 (L3) per book. A four-book z990 can have up to 256 GB memory. The
Storage Controller, shown as SCC CNTLR in Figure2-12 on page 40, acts as a cross-point
switch between processor units (PUs), Memory Controllers (MSCs), and Memory Bus
Adapters (MBAs).
The SD chips, shown as SCD in Figure2-12, also incorporate a Memory Coherent Controller
(MCC) function.
Each PU chip has its own 512 KB Cache Level 1 (L1), split into 256 KB for data and 256 KB
for instructions. The L1 cache is designed as a store-through cache, meaning that altered
data is also stored to the next level of memory (L2 cache). The z990 models A08, B16, C24,
and D32 use the CMOS 9SG PU chips running at 0.83 ns.
The MCC controls a large 32MB L2 cache, and is responsible for the interbook
communication in a ring topology connecting up to four books through two concentric loops,
called the ring structure. The MCC optimizes cache traffic and will not look for cache hits in
other books when it knows that all resources of a given logical partition are available in the
same book.
The L2 cache is the aggregate of all cache space on the SD chips, resulting in a 32 MB L2
cache per book. The SC chip (SCC) controls the access and storing of data in the four SD
chips. The L2 cache is shared by all PUs within a book and shared across books through the
ring topology, providing the communication between L2 caches across books in systems with
more than one book installed; the L2 has a store-in buffer design.
The interface between the L2 cache and processor memory (L3) is accomplished by four
high-speed memory buses and controlled by the memory controllers (MSC). Storage access
is interleaved between the storage cards, which tends to equalize storage activity across the
cards. Each memory card has two ports that each have a maximum bandwidth of 8 GB/sec.
Each port contains a control and a data bus, in order to further reduce any contention by
separating the address and command from the data bus.
The memory cards support store protect key caches to match the key access bandwidth with
that of the memory bandwidth.
The logical book structure is shown in Figure 2-12.