Manuals / AMD / Computer Equipment / Computer Hardware

AMD 64 manual Appendix A, A.1 Description of the Buffer Queues, C0 C1

Models: 64

1 48

Download 48 pages 55.63 Kb

Page 39

Appendix A

40555 Rev. 3.00 June 2006	Performance Guidelines for AMD Athlon™ 64 and AMD Opteron™
	ccNUMA Multiprocessor Systems

Appendix A

The following sections provide additional explanatory information on topics discussed in the previous sections of this document.

A.1 Description of the Buffer Queues

Figure 16 shows the internal resources in each Quartet node. The memory controller (MCT), the System Request Interface (SRI) and the Crossbar (XBar) on each node all have internal buffers that are used to queue transmitted transaction packets. Physically each of these units only has two kinds of buffers, command and data.

C0

C1

HT = HyperTransport™

cHT = coherent HyperTransport

Figure 16. Internal Resources Associated with a Quartet Node

Consider the buffers that come into play in the interface between the XBar and the HyperTransport™ links.

Each node has an incoming and outgoing coherent HyperTransport link on its XBar to every other node but one in the system. Node 0 has an outgoing link that allows it to send data from node 0 to node 1. Likewise node 0 has an incoming link from node 1 to node 0. The two links together can be considered as one bidirectional link.

Each node has HyperTransport buffers in its XBar that are used to queue up the packets that are going to be sent on the outgoing link. The sending node does not send the packets until the receiving node is ready to receive them.

Now Consider the buffers that come into play in the interface between the XBar and the MCT. Packets to be transmitted from the XBar to the MCT are queued in the ‘XBar-to-MCT” buffers.

Appendix A

39

Image 39

AMD 64 manual Appendix A, A.1 Description of the Buffer Queues, Internal Resources Associated with a Quartet Node, C0 C1

Contents

Application Note Issue Date JunePerformance Guidelines for AMD Athlon 64 and AMD Opteron ccNUMA Multiprocessor SystemsTrademarks 2006 Advanced Micro Devices, Inc. All rights reservedChapter 3 Analysis and Recommendations ContentsChapter 2 Experimental Setup ContentsA.2.1 What Resources Are Used When a Single Read-Only or List of Figures List of Figures40555 Rev. 3.00 June List of FiguresPerformance Guidelines for AMD Athlon 64 and AMD Opteron ccNUMA Multiprocessor SystemsPerformance Guidelines for AMD Athlon 64 and AMD Opteron Revision HistoryRevision History ccNUMA Multiprocessor Systems40555 Rev. 3.00 June Revision HistoryPerformance Guidelines for AMD Athlon 64 and AMD Opteron ccNUMA Multiprocessor SystemsChapter 1 Introduction ChapterIntroduction 1.1 Related Documents 10 http//msdn2.microsoft.com/en-us/library/ms186255SQL.90.aspxIntroduction 14 http//msdn2.microsoft.com/en-us/library/tt15eb9t.aspx ChapterIntroduction Chapter IntroductionPerformance Guidelines for AMD Athlon 64 and AMD Opteron ccNUMA Multiprocessor SystemsChapter Chapter 2 Experimental SetupExperimental Setup 2.1 System UsedExperimental Setup Figure 1. Quartet TopologyChapter 2.2 Synthetic TestFigure 2. Internal Resources Associated with a Quartet Node Experimental SetupExperimental Setup Data Access Rate QualifiersChapter 2.3 Reading and Interpreting Test Graphs2.3.1 X-Axis Display Experimental Setup2.3.2 Labels Used 2.3.3 Y-Axis DisplayExperimental Setup Analysis and Recommendations Chapter 3 Analysis and Recommendations3.1 Scheduling Threads 3.1.1 Multiple Threads-Independent DataAnalysis and Recommendations 3.2 Data Locality Considerations3.1.2 Multiple Threads-Shared Data 3.1.3 Scheduling on a Non-Idle System100% ChapterAnalysis and Recommendations 2 HopAnalysis and Recommendations 3.2.1 Keeping Data Local by Virtue of first TouchAnalysis and Recommendations ChapterAnalysis and Recommendations afterwords no longer needs the data structure and if only one of the worker threads needs the data structure. In other words, the data structure is not truly shared between the worker threads3.4.1 Myth All Equal Hop Cases Take Equal Time Threads access local data3.3 Avoid Cache Line Sharing 3.4 Common Hop Myths DebunkedAnalysis and Recommendations Threads firing at each other crossfireNode 0 Core Node 1 Core Node 2 Core Node 3 Core ChapterAnalysis and Recommendations Analysis and Recommendations Next, we increase the number of background threads to six, running on216% 3.4.2 Myth Greater Hop Distance Always Means Slower TimeChapter Analysis and RecommendationsAnalysis and Recommendations Both threads access memory on node0 hop-0 hop case for the write-only threads ChapterAnalysis and Recommendations Analysis and Recommendations In addition, three background threads are running on nodes 1, 2 andHigh Total Time for both threads write-write ChapterAnalysis and Recommendations Medium Total Time for both threads write-write158% 158% 3.5 LocksAnalysis and Recommendations 147%Analysis and Recommendations ChapterChapter Analysis and RecommendationsPerformance Guidelines for AMD Athlon 64 and AMD Opteron ccNUMA Multiprocessor SystemsChapter 4 Conclusions ConclusionsChapter Conclusions Data placement tools can also come in handy when a thread needs more data than the amount of physical memory available on a node. Certain OSs also allow data migration with these tools or API. Using this feature, data can be migrated from the node where it was first touched to the node where it is subsequently accessed. There is a cost associated with this migration and it is not advised to use it frequently. For additional details on the tools and APIs offered by various OS for thread and memory placement refer to Section A.7 on pageAppendix A Appendix AA.1 Description of the Buffer Queues Figure 16. Internal Resources Associated with a Quartet NodePage Appendix A A.2.3 What Role Do Buffers Play in the Throughput Observed?0 Hop-1 Hop Case on an Idle System for Write- Only Threads? A.4 Why Is 0 Hop-0 Hop Case Slower Than theA.5 Why Is 0 Hop-1 Hop Case Slower Than Controlling Process and Thread Affinity A.7.1 Support Under LinuxAppendix A A.7.2 Support under SolarisA.7.3 Support under Microsoft Windows Controlling Memory AffinityA.8.1 Support under Linux A.8.2 Support under SolarisA.8.3 Support under Microsoft Windows Appendix A A.8.4 Node Interleaving Configuration in the BIOS40555 Rev. 3.00 June Performance Guidelines for AMD Athlon 64 and AMD OpteronccNUMA Multiprocessor Systems Appendix A