40555 Rev. 3.00 June 2006 | Performance Guidelines for AMD Athlon™ 64 and AMD Opteron™ |
| ccNUMA Multiprocessor Systems |
Controlling Memory Affinity
Both numactl and libnuma library functions can be used to set memory affinity[5]. Memory affinity set by tools like numactl applies to all the data accessed by the entire program (including child processes). Memory affinity set by libnuma or other library functions can be made to apply only to
specific data as determined by the program.
Both numactl and the libnuma API can be used to set a preferred memory affinity instead of forcibly binding it. In this case the binding specified is a hint to the OS; the OS may choose not to adhere to it.
At a high level, normal first touch binding, explicit binding and preferred binding are all available as memory policies on Linux.
By default, when none of the tools/API is used, Linux uses the first touch binding policy for all data. Once memory is bound, either by the OS, or by using the tools/API, the memory will normally remain resident on that node for its lifetime.
A.7.2 Support under Solaris
Sun Solaris provides several tools and API's for influencing thread/process and memory affinity[6].
Solaris provides a command line tool called pbind to set process affinity. There is also a shared library called liblgrp that provides an API that a program can call to set thread affinity.
Solaris provides a memory placement API to affect memory placement. A program can call the madvise( ) function to provide hints to the OS as to the memory policy to use. This API does not
allow binding of memory to an explicit node or set of nodes specified on the command line or in the program. But there are several policies other than the first touch policy that can be used.
For example, a thread can use madvise to migrate the data it needs to the node where it runs, instead of leaving it on a different node, on which it was first touched by another thread. There is, naturally, a cost associated with the migration.
Solaris provides a library called madv.so.1 which can interpose on memory allocation system calls and call the madvise function internally for the memory policy.
By default, Solaris uses the first touch binding policy for data that is not shared. Once memory is bound to a node it normally remains resident on that node for its lifetime.
Sun is also working on supporting several command line tools to control thread and memory placement. These are expected to be integrated in the upcoming versions of Solaris, but experimental versions are currently available[7].
A.7.3 Support under Microsoft® Windows®
In the Microsoft Windows environment, the function to bind a thread on particular core or cores is SetThreadAffinityMask( ). The function to run all threads in a process on particular core or cores is SetProcessAffinityMask( )[8].
Appendix A | 45 |
