IBM pSeries manual

IBM ~ pSeries High Performance Switch

Tuning and Debug Guide

Version 1.0

April 2005

IBM Systems and Technology Group

Cluster Performance Department

Poughkeepsie, NY

Contents

Main IBM ~ pSeries High Performance Switch Contents Page 1.0 Introduction 2.0 Tunables and settings for switch software 2.1 MPI tunables for Parallel Environment 2.1.1 MP_EAGER_LIMIT 2.1.2 MP_POLLING_INTERVAL and MP_RETRANSMIT_INTERVAL 2.1.3 MP_REXMIT_BUF_SIZE and MP_REXMIT_BUF_CNT 2.1.4 MEMORY_AFFINITY 2.1.5 MP_TASK_AFFINITY 2.1.6 MP_CSS_INTERRUPT 2.2 MPI-IO 2.3 chgsni command 3.0 Tunables and settings for AIX 5L 3.1 IP tunables 3.2 File cache 3.3 svmon and vmstat commands 3.3.1 svmon 3.3.2 vmstat 3.4 Large page sizing Page 3.5 Large pages and IP support 3.6 Memory affinity for a single LPAR 3.7 Amount of memory available 3.8 Debug settings in the AIX 5L kernel 4.0 Daemon configuration 4.1 RSCT daemons 4.2 LoadLeveler daemons 4.2.1 Reducing the number of daemons running 4.2.2 Reducing daemon communication and placing daemons on a switch 4.2.3 Reducing logging 4.3 Settings for AIX 5L threads 4.4 AIX 5L mail, spool, and sync daemons 4.5 Placement of POE managers and LoadLeveler scheduler 5.0 Debug settings and data collection tools 5.1 lsattr tuning driver_debug 0 ip_trc_lvl 0x1111 5.1.1 driver_debug setting 5.3 Affinity LPARs 5.4 Small Real Mode Address Region on HMC GUI 64GB 61.5GB 128GB 120GB 256GB 240GB 512GB 495GB 5.5 Deconfigured L3 cache 5.7 errpt command 5.8 HMC error logging 5.9 Multiple versions of MPI libraries 5.10 MP_PRINTENV 5.11 MP_STATISTICS 5.12 Dropped switch packets 5.12.1 Packets dropped because of a software problem on an endpoint Page 5.12.2 Packets dropped in the ML0 interface 5.12.3 Packets dropped because of a hardware problem on an endpoint 5.12.4 Packets dropped in the switch hardware 5.13 MP_INFOLEVEL 5.14 LAPI_DEBUG_COMM_TIMEOUT 5.15 LAPI_DEBUG_PERF 5.16 AIX 5L trace for daemon activity 6.0 Conclusions and summary 7.0 Additional reading 7.1 HPS documentation