Escali 4.4 manual Example, Features

Page 52

0:MPI_Bcast root: 0 Id: 0 my_count = 32768

0:MPI_Scatter Id: 1

1:MPI_Init

1:MPI_Comm_rank Rank: 1

1:MPI_Comm_size Size: 2

1:MPI_Bcast root: 0 Id: 0 my_count = 32768

1:MPI_Scatter Id: 1

1:MPI_Reduce Sum root: 0 Id: 2

1:MPI_Bcast root: 0 Id: 3

0:MPI_Reduce Sum root: 0 Id: 2

0:MPI_Bcast root: 0 Id: 3

1:MPI_Gather Id: 4

1:MPI_Keyval_free

0:MPI_Gather Id: 4

0:MPI_Keyval_free

If more information is needed the arguments to SCAMPI_TRACE can be enhanced to request more information. The option “-f arg;timing” requests a list of the arguments given to each MPI call, including message size ( useful information when evaluating interconnect performance).

Example:

%SCAMPI_TRACE="-f arg;timing" mpimon ./kollektive-8 ./uf256-8.pgm -- n1 2

0:+-0.951585 s 951.6ms MPI_Init

0:+0.000104 s 3.2us MPI_Comm_rank Rank: 0

0:+0.000130 s 1.7us MPI_Comm_size Size: 2

0:+0.038491 s 66.3us MPI_Bcast root: 0 sz: 1 x 4 = 4 Id: 0 my_count = 32768

0:+0.038634 s 390.0us MPI_Scatter Id: 1

1: +-1.011783 s

1.0s MPI_Init

1:+0.000100 s 3.8us MPI_Comm_rank Rank: 1

1:+0.000129 s 1.7us MPI_Comm_size Size: 2

1:+0.000157 s 69.6us MPI_Bcast root: 0 sz: 1 x 4 = 4 Id: 0 my_count = 32768

1:+0.000300 s 118.7us MPI_Scatter Id: 1

0:+0.039267 s 38.8ms MPI_Reduce Sum root: 0 sz: 1 x 4 = 4 Id: 2

0:+0.078089 s 18.8us MPI_Bcast root: 0 sz: 1 x 8 = 8 Id: 3

1:+0.000641 s 56.2us MPI_Reduce Sum root: 0 sz: 1 x 4 = 4 Id: 2

1:+0.000726 s 113.6us MPI_Bcast root: 0 sz: 1 x 8 = 8 Id: 3

1:+0.002547 s 99.0us MPI_Gather Id: 4

0:+0.079834 s 579.5us MPI_Gather Id: 4

1:+0.002758 s 26.4us MPI_Keyval_free

0:+0.113244 s 1.4us MPI_Keyval_free

There are a number of parameters for selecting only a subset, either by limiting the number of calls and intervals as described above under ‘Timing’ , or selecting or excluding just some MPI calls.

4.2.2 Features

The "-b" option is useful when trying to pinpoint which MPI-call has been started but not completed (i.e. are deadlocked). The "-s/-S/-c/-C"-optionsalso offer useful support for an application that runs well for a longer period and then stop, or for examining some part of the execution of the application.

Scali MPI Connect Release 4.4 Users Guide

40

Image 52
Contents Scali MPI ConnectTM Users Guide Acknowledgement Copyright 1999-2005 Scali AS. All rights reservedScali Bronze Software Certificate II Software License Terms Commencement MaintenanceGrant of License Support License ManagerSub-license and distribution Export RequirementsSCALI’s Obligations LICENSEE’s ObligationsTitle to Intellectual Property Rights TransferWarranty of Title and Substantial Performance Compliance with LicensesLimitation on Remedies and Liabilities Scali MPI Connect Release 4.4 Users Guide ViiProprietary Information MiscellaneousGoverning Law Scali MPI Connect Release 4.4 Users Guide Table of contents Profiling with Scali MPI Connect Appendix a Example MPI code Scali MPI Connect Release 4.4 Users Guide Chapter Scali MPI Connect product contextScali mailing lists SMC FAQ SMC release documents Problem reportsSupport Platforms supportedHow to read this guide Acronyms and abbreviationsLicensing FeedbackNIC Typographic conventions Terms and conventionsGUI style font Typographic conventions Description of Scali MPI Connect Scali MPI Connect componentsSMC network devices Direct Access Transport DAT Network devicesShared Memory Device Ethernet DevicesUsing detstat Using detctl3.2 DET Myrinet Infiniband4.1 GM 5.1 IB6 SCI Communication protocols on DAT-devicesChannel buffer Eagerbuffering protocol Inlining protocolTransporter protocol Support for other interconnects MPI-2 FeaturesZerocopy protocol Scali MPI Connect Release 4.4 Users Guide MPI-2 Features Setting up a Scali MPI Connect environment Compiling and linkingScali MPI Connect environment variables RunningCompiler support Linker flagsRunning Scali MPI Connect programs Naming conventionsBasic usage Mpimon monitor programIdentity of parallel processes Controlling options to mpimon Standard inputStandard output Program specHow to provide options to mpimon Network optionsMpirun wrapper script Mpirun usageRunning with tcp error detection Tfdr Suspending and resuming jobsRunning with dynamic interconnect failover capabilities Part partDebugging and profiling Debugging with a sequential debuggerUsing built-in segment protect violation handler Built-in-tools for debuggingAssistance for external profiling Debugging with Etnus TotalviewCommunication resources on DAT-devices Controlling communication resourcesChannelinlinethreshold size to set threshold for inlining Using MPIIsend, MPIIrecv Using MPIBsendGood programming practice with SMC Matching MPIRecv with MPIProbeError and warning messages User interface errors and warningsFatal errors Unsafe MPI programsMpimon options Prefix Giving numeric values to mpimonPostfix Scali MPI Connect Release 4.4 Users Guide Profiling with Scali MPI Connect ExampleUsing Scali MPI Connect built-in trace TracingAbsRank MPIcallcommNamerankcall-dependant-parameters where +relSecs S eTime whereFeatures ExampleUsing Scali MPI Connect built-in timing TimingMPIcallDcallsDtimeDfreq TcallsTtimeTfreq Using the scanalyze Commrank recv from fromworldFromcommonFieldsCommrank send to toworldTocommonFields where Count!avrLen!zroLen!inline!eager!transporter! whereFor timing Using SMCs built-in CPU-usage functionality This produces the following reportScali MPI Connect Release 4.4 Users Guide Tuning communication resources Automatic buffer managementHow to optimize MPI performance BenchmarkingCaching the application program on the nodes First iteration is very slowCollective operations Memory consumption increase after warm-upFinding the best algorithm Programs in the ScaMPItst package Appendix aImage contrast enhancement Scali MPI Connect Release 4.4 Users Guide File format OriginalWhy does not my program start to run? When things do not work troubleshootingAppendix B Why does my program terminate abnormally? Why can I not start mpid?General problems Per node installation of Scali MPI Connect Appendix CInstall Scali MPI Connect for TCP/IP Install Scali MPI Connect for Direct EthernetInstall Scali MPI Connect for Myrinet ExampleInstall Scali MPI Connect for Infiniband Install Scali MPI Connect for SCIInstall and configure SCI management software License optionsTroubleshooting Network providers Uninstalling SMCScali kernel drivers Troubleshooting 3rdparty DAT providers Troubleshooting the GM providerScali MPI Connect Release 4.4 Users Guide Bracket expansion Appendix D Bracket expansion and groupingGrouping Scali MPI Connect Release 4.4 Users Guide Appendix E Related documentationScali MPI Connect Release 4.4 Users Guide List of figures Scali MPI Connect Release 4.4 Users Guide Index Transporter protocolSSP