Manuals / AMD / Computer Equipment / Computer Hardware

AMD 64 manual A.8.1 Support under Linux, A.8.2 Support under Solaris

Models: 64

1 48

Download 48 pages 55.63 Kb

Page 46

A.8.1 Support under Linux®

Performance Guidelines for AMD Athlon™ 64 and AMD Opteron™	40555 Rev. 3.00 June 2006
ccNUMA Multiprocessor Systems

The function to set memory affinity for a thread is VirtualAlloc( )[9]. This function gives the developer the choice to bind memory immediately on allocation or to defer binding until first touch.

Although there are no command-line tools for thread/process and memory placement, several Microsoft Enterprise products provide NUMA support and configurability, such as SQL Server 2005 [10] and IIS [11].

If an application relies on heaps in Windows, we recommend using a low fragmentation heap (LFH) and using a local heap instead of a global heap[12][13].

By default, Windows uses the first touch binding policy for all data. Once memory is bound to a node, it normally resides on that node for its lifetime.

A.8 Tools and APIs for Node Interleaving in Various OSs for AMD64 ccNUMA Multiprocessor Systems

This section discusses tools and APIs available for performing node interleaving under various operating systems.

A.8.1 Support under Linux®

Linux provides several ways for an application to use node interleaving [5].

•numactl is a command line tool, which is used for node interleaving all memory accessed by a program across a set of chosen nodes.

For example, to interleave all memory accessed by program foobar on nodes 0 and 1, use:

numactl --interleave=0x03 foobar

•libnuma offers several functions a program can use to interleave a given memory region across a set of chosen nodes.

Linux only supports the round robin node interleaving policy.

A.8.2 Support under Solaris

Solaris offers an API called madvise, which can be usedwith the MADV_ACCESS_MANY flag to tell the OS to use a memory policy that causes the OS to bind memory randomly across the nodes. This offers behavior similar to the round robin node interleaving of memory offered by Linux.

This random policy is the default memory placement policy used by Solaris for shared memory.

A.8.3 Support under Microsoft® Windows®

Microsoft Windows does not offer node interleaving.

46	Appendix A

Image 46

AMD 64 manual A.8.1 Support under Linux, A.8.2 Support under Solaris, A.8.3 Support under Microsoft Windows

Contents

AMD Opteron ccNUMA Multiprocessor Systems Issue Date JunePerformance Guidelines for AMD Athlon 64 and Application Note2006 Advanced Micro Devices, Inc. All rights reserved TrademarksContents ContentsChapter 2 Experimental Setup Chapter 3 Analysis and RecommendationsA.2.1 What Resources Are Used When a Single Read-Only or List of Figures List of FiguresccNUMA Multiprocessor Systems List of FiguresPerformance Guidelines for AMD Athlon 64 and AMD Opteron 40555 Rev. 3.00 JuneccNUMA Multiprocessor Systems Revision HistoryRevision History Performance Guidelines for AMD Athlon 64 and AMD OpteronccNUMA Multiprocessor Systems Revision HistoryPerformance Guidelines for AMD Athlon 64 and AMD Opteron 40555 Rev. 3.00 JuneChapter Chapter 1 IntroductionIntroduction 1.1 Related Documents IntroductionChapter 14 http//msdn2.microsoft.com/en-us/library/tt15eb9t.aspxIntroduction ccNUMA Multiprocessor Systems IntroductionPerformance Guidelines for AMD Athlon 64 and AMD Opteron Chapter2.1 System Used Chapter 2 Experimental SetupExperimental Setup ChapterFigure 1. Quartet Topology Experimental SetupExperimental Setup 2.2 Synthetic TestFigure 2. Internal Resources Associated with a Quartet Node ChapterData Access Rate Qualifiers Experimental SetupExperimental Setup 2.3 Reading and Interpreting Test Graphs2.3.1 X-Axis Display Chapter2.3.3 Y-Axis Display 2.3.2 Labels UsedExperimental Setup 3.1.1 Multiple Threads-Independent Data Chapter 3 Analysis and Recommendations3.1 Scheduling Threads Analysis and Recommendations3.1.3 Scheduling on a Non-Idle System 3.2 Data Locality Considerations3.1.2 Multiple Threads-Shared Data Analysis and Recommendations2 Hop ChapterAnalysis and Recommendations 100%3.2.1 Keeping Data Local by Virtue of first Touch Analysis and RecommendationsChapter Analysis and Recommendationsafterwords no longer needs the data structure and if only one of the worker threads needs the data structure. In other words, the data structure is not truly shared between the worker threads Analysis and Recommendations3.4 Common Hop Myths Debunked Threads access local data3.3 Avoid Cache Line Sharing 3.4.1 Myth All Equal Hop Cases Take Equal TimeThreads firing at each other crossfire Analysis and RecommendationsChapter Node 0 Core Node 1 Core Node 2 Core Node 3 CoreAnalysis and Recommendations Next, we increase the number of background threads to six, running on Analysis and RecommendationsAnalysis and Recommendations 3.4.2 Myth Greater Hop Distance Always Means Slower TimeChapter 216%Both threads access memory on node Analysis and RecommendationsChapter 0 hop-0 hop case for the write-only threadsAnalysis and Recommendations In addition, three background threads are running on nodes 1, 2 and Analysis and RecommendationsMedium Total Time for both threads write-write ChapterAnalysis and Recommendations High Total Time for both threads write-write147% 3.5 LocksAnalysis and Recommendations 158% 158%Chapter Analysis and RecommendationsccNUMA Multiprocessor Systems Analysis and RecommendationsPerformance Guidelines for AMD Athlon 64 and AMD Opteron ChapterConclusions Chapter 4 ConclusionsChapter Data placement tools can also come in handy when a thread needs more data than the amount of physical memory available on a node. Certain OSs also allow data migration with these tools or API. Using this feature, data can be migrated from the node where it was first touched to the node where it is subsequently accessed. There is a cost associated with this migration and it is not advised to use it frequently. For additional details on the tools and APIs offered by various OS for thread and memory placement refer to Section A.7 on page ConclusionsFigure 16. Internal Resources Associated with a Quartet Node Appendix AA.1 Description of the Buffer Queues Appendix APage A.2.3 What Role Do Buffers Play in the Throughput Observed? Appendix AA.4 Why Is 0 Hop-0 Hop Case Slower Than the 0 Hop-1 Hop Case on an Idle System for Write- Only Threads?A.5 Why Is 0 Hop-1 Hop Case Slower Than A.7.1 Support Under Linux Controlling Process and Thread AffinityControlling Memory Affinity A.7.2 Support under SolarisA.7.3 Support under Microsoft Windows Appendix AA.8.2 Support under Solaris A.8.1 Support under LinuxA.8.3 Support under Microsoft Windows A.8.4 Node Interleaving Configuration in the BIOS Appendix AAppendix A Performance Guidelines for AMD Athlon 64 and AMD OpteronccNUMA Multiprocessor Systems 40555 Rev. 3.00 June