Escali 4.4 manual

Contents

Main Acknowledgement SCALI BRONZE SOFTWARE CERTIFICATE Page Page Page Page Page Page Page Page Page Table of contents Chapter 1 Introduction ....................................................................5 Chapter 2 Description of Scali MPI Connect ...................................11 Chapter 3 Using Scali MPI Connect ................................................21 Chapter 4 Profiling with Scali MPI Connect.................................... 37 Chapter 5 Tuning SMC to your application ..................................... 47 Page Page Chapter 1 Introduction 1.1 Scali MPI Connect product context 1.2 Support 1.2.1 Scali mailing lists 1.2.2 SMC FAQ 1.2.3 SMC release documents 1.2.4 Problem reports 1.3 How to read this guide 1.4 Acronyms and abbreviations Page 1.5 Terms and conventions 1.6 Typographic conventions Page Chapter 2 Description of Scali MPI Connect 2.1 Scali MPI Connect components 2.2 SMC network devices 2.2.1 Network devices 2.2.1.1 Direct Access Transport (DAT) 2.2.2 Shared Memory Device 2.2.3 Ethernet Devices 2.2.3.1 TCP 2.2.3.2 DET 2.2.3.3 Using detctl 2.2.3.4 Using detstat 2.2.4 Myrinet 2.2.4.1 GM 2.2.5 Infiniband 2.2.5.1 IB 2.2.6 SCI 2.3 Communication protocols on DAT-devices 2.3.1 Channel buffer Page 2.3.5 Zerocopy protocol 2.4 Support for other interconnects 2.5 MPI-2 Features Page Page Chapter 3 Using Scali MPI Connect 3.1 Setting up a Scali MPI Connect environment 3.1.1 Scali MPI Connect environment variables 3.2 Compiling and linking 3.2.1 Running 3.2.2 Compiler support 3.2.3 Linker flags 3.2.4 Notes on Compiling and linking on AMD64 and EM64T 3.2.5 Notes on Compiling and linking on Power series 3.2.6 Notes on compiling with MPI-2 features 3.3 Running Scali MPI Connect programs 3.3.1 Naming conventions 3.3.2 mpimon - monitor program 3.3.2.1 Basic usage 3.3.2.2 Identity of parallel processes 3.3.2.3 Controlling options to mpimon 3.3.2.4 Standard input 3.3.2.5 Standard output 3.3.2.6 How to provide options to mpimon 3.3.2.7 Network options 3.3.3 mpirun - wrapper script 3.3.3.1 mpirun usage 3.4 Suspending and resuming jobs 3.5 Running with dynamic interconnect failover capabilities 3.6 Running with tcp error detection - TFDR 3.7 Debugging and profiling 3.7.1 Debugging with a sequential debugger 3.7.2 Built-in-tools for debugging 3.7.2.1 Using built-in segment protect violation handler 3.7.3 Assistance for external profiling 3.7.4 Debugging with Etnus Totalview 3.8 Controlling communication resources 3.8.1 Communication resources on DAT-devices 3.9 Good programming practice with SMC 3.9.1 Matching MPI_Recv() with MPI_Probe() 3.9.2 Using MPI_Isend(), MPI_Irecv() 3.9.3 Using MPI_Bsend() 3.9.4 Avoid starving MPI-processes - fairness 3.10 Error and warning messages 3.10.1 User interface errors and warnings 3.10.2 Fatal errors 3.11 Mpimon options 3.11.1 Giving numeric values to mpimon Page Chapter 4 Profiling with Scali MPI Connect 4.1 Example 4.2 Tracing Page Page 4.3 Timing Page 4.4 Using the scanalyze Page 4.5 Using SMC's built-in CPU-usage functionality Page Chapter 5 Tuning SMC to your application 5.1 Tuning communication resources 5.2 How to optimize MPI performance 5.3 Benchmarking 5.4 Collective operations Page Appendix A Example MPI code A-1 Programs in the ScaMPItst package A-2 Image contrast enhancement Page Page Appendix B Troubleshooting B-1 When things do not work - troubleshooting B-1.1 Why does not my program start to run? B-1.2 Why can I not start mpid? B-1.3 Why does my program terminate abnormally? B-1.4 General problems Appendix C Install Scali MPI Connect C-1 Per node installation of Scali MPI Connect C-2 Install Scali MPI Connect for TCP/IP C-3 Install Scali MPI Connect for Direct Ethernet C-4 Install Scali MPI Connect for Myrinet C-5 Install Scali MPI Connect for Infiniband C-6 Install Scali MPI Connect for SCI C-7 Install and configure SCI management software C-8 License options C-9 Scali kernel drivers C-10 Uninstalling SMC C-11 Troubleshooting Network providers C-11.1 Troubleshooting 3rdparty DAT providers C-11.2 Troubleshooting the GM provider Page Appendix D Bracket expansion and grouping D-1 Bracket expansion D-2 Grouping Page Appendix E Related documentation Page Page Page Index