IBM pSeries manual Large pages and IP support, Memory affinity for a single Lpar

3.5 Large pages and IP support

One of the most important ways to improve IP performance on the HPS is to ensure that large pages are enabled. Large pages are required to allocate a number of large pages which will used by the HPS IP driver at boot time.

Each snX needs one large page for the IP FIFO, plus the number of send pools and receive pools shared among all adapters. Here is the formula for the number of large pages, assuming that the send pool and receive pool each need two pages.

(N_adapters*(1 + 2send pools + 2receive pools))

To check whether the driver is using large pages, run the following command:

/usr/sbin/ifsn_dump -a grep use_lg_pg

If large pages are being used, you should see this result:

use_lg_pg 0x00000001

If large pages are not used for the IP pools, the ifsn_dump -rtraces (discussed later) report this with the following messages:

[02]1099351956s 0831707096ns 0x000c sn_setup_if_env: large page check failed! lgpg_size=0x1000000, lgpg_cnt=[. . .]

[02]1099351956s 0831708002ns 0x000d sn_setup_if_env: large page check failed! num_pages=0x4, lgpg_numfrb=[. . .]

3.6 Memory affinity for a single LPAR

If you are running with one big LPAR containing all processors on a p690 machine, you need to ensure that memory affinity is set correctly. To do this using vmo, set memory_affinity = 1.

This works with the AIX 5L shell variable MEMORY_AFFINITY and with the MPI variable MP_TASK_AFFINITY described earlier.

3.7 Amount of memory available

To properly size the number of large pages on a system, or to determine the largest problem size that an MPI task can run, you need to determine the amount of configured memory in your LPAR. To do this, run the following command:

lsattr -E -l sys0 -a realmem

To find the actual physical real memory installed on you CEC or LPAR, run the following command:

lscfg -vp grep GB

pshpstuningguidewp040105.doc Page 15