HP B- Accelerators Linux manual Numa configuration, Introduction to Numa architecture

NUMA configuration

Introduction to NUMA architecture

Servers with NUMA (Non-Uniform Memory Access) architecture require special installation instructions in order to maximize IO Accelerator device performance. These servers include the HP DL580 and the HP DL980 server.

On servers with NUMA architecture, during system boot, the BIOS on some systems will not distribute PCIe slots evenly among the NUMA nodes. Each NUMA node contains multiple CPUs. This imbalanced distribution means that, during high workloads, half or more of the CPUs will remain idle while the rest are 100% utilized. To prevent this imbalance, you must manually assign IO Accelerator devices equally among the available NUMA nodes.

Configuring the IO Accelerator devices for servers FIO_AFFINTIY parameter with the fio-config

with NUMA architecture requires the use of the utility.

NUMA node override parameter

The numa_node_override parameter is a list of <affinity specification> couplets that specify the affinity settings of all devices in the system. Each item in the couplet is separated by a colon, and each couplet set is separated by a comma.

Syntax

numa_node_override=<affinity specification>[,<affinity specification>...] Where each <affinity specification> has the following syntax: <fct-number>:<node-number>

Simple example

numa_node_override=fct4:1,fct5:0,fct7:2,fct9:3

has the effect of creating:

Device	Node/group	Process or affinity
fct4	node 1	all processors in node 1
fct5	node 0	all processors in node 0
fct7	node 2	all processors in node 2
fct9	node 3	all processors in node 3

Advanced configuration example

This example server has four NUMA nodes with eight hyper-threaded cores per node (16 logical processors per node, a total of 64 logical processors in the system). This system also uses the expansion configuration and has 11 PCIe expansion slots. During system boot, the system BIOS assigns PCIe slots 1-6 to NUMA node 2 and PCIe slots 7-11 to NUMA node 0. NUMA nodes 1 and 3 have no assigned PCIe slots. This

NUMA configuration 57