IBM HPSS manual

HPSS

Installation Guide

High Performance Storage System

Release 6.2

July 2008 (Revision 2.0)

HPSS Installation Guide July 2008

Release 6.2 (Revision 2.0) 1

Contents

Main HPSS Installation Guide July 2008 Release 6.2 (Revision 2.0) 2 Table of Contents Release 6.2 (Revision 2.0) 4 Release 6.2 (Revision 2.0) 5 Release 6.2 (Revision 2.0) 6 Page Page Release 6.2 (Revision 2.0) 9 Page Page List of Figures List of Tables Preface Release 6.2 (Revision 2.0) 14 Page 1.1.9. Additional hpssadm operations 1.1.10. Additional Librar y and Device Support 1.1.11. SAN Virtual Volume ID Mapping 1.1.12. Drive Pools 1.1.13. FTP Enhancement 1.1.14. mkhpss Enhancement 1.1.15. DB2 Monitoring 1.1.16. File Family enhance ments 1.1.18. Mover Enhancement 1.2. Retired Features 1.3. Deferred Features 1.4. HPSS Changes 1.4.1. Documentation Organ ization Changes 1.4.3. DMAP Gateway Change s 1.4.3.1. Creating an XDSM Fileset 1.4.3.2. Viewing DMAP Gatewa y XDSM Fileset Information 1.4.3.3. Viewing Core Server XDSM Fileset Information HPSS Installation Guide July 2008 Release 6.2 (Revision 2.0) 20 1.4.4. SSM Changes 1.4.4.1. Changes Affectin g Sites Upgrading Direct ly from 4.5 Page Page May 2, 2005 11:26:13 AM Major PVL Bad things happened Page Page 1.4.4.2. Changes Affecting Sites Upgrading from 5.1 Page Page Page Page Page Page Page Chapter 2. HPSS Basics 2.1. Introduction 2.2. HPSS Capabilities 2.2.1. Network-centered Archit ecture 2.2.2. High Data Transfer Rate 2.2.3. Parallel Operation 2.2.4. Based on Standard Components 2.2.5. Data Integrity Th rough Transaction Managemen t Release 6.2 (Revision 2.0) 36 2.2.6. Multiple Hierarchie s and Classes of Service s 2.2.7. Storage Subsystem s 2.3. HPSS Components 2.3.1. HPSS Files, Filese ts, Volumes, Storage Seg ments and Related Metadata Page 2.3.2. HPSS Servers Page Page Release 6.2 (Revision 2.0) 43 2.3.3. HPSS Storage Subsys tems 2.3.4. HPSS Infrastructu re Page 2.3.5. HPSS User Interfac es 2.3.6. HPSS Management In terfaces Release 6.2 (Revision 2.0) 46 2.3.7. HPSS Policy Module s 2.4. HPSS Hardware Platf orms 2.4.1. Server Platform s 2.4.2. Client Platform s Table 1. HPSS C lient Interface and M over Platforms 2.4.3. Mover Platforms Release 6.2 (Revision 2.0) 49 Page Chapter 3. HPSS Planning 3.1. Overview 3.1.1. HPSS System Architectu re Page Page 3.1.3. Purchasing Hardware and Softwa re 3.1.4. HPSS Operational P lanning 3.1.5. HPSS Deployment Pla nning 3.2. Requirements and I ntended Uses for HPSS 3.2.1. Storage System Capacity 3.2.2. Required Throughp uts 3.2.3. Load Characteriza tion 3.2.4. Usage Trends 3.2.5. Duplicate File Pol icy 3.2.6. Charging Policy Release 6.2 (Revision 2.0) 57 3.2.7. Security 3.2.7.1. Cross Realm Acce ss 3.2.8. High Availability Option 3.3. Prerequisite Softwar e Considerations 3.3.1. Prerequisite Soft ware Overview 3.3.1.1. DB2 3.3.1.2. Kerberos 3.3.1.3. LDAP and IBM Kerbero s 3.3.1.4. Java 3.3.2. Prerequisite Summary By HPSS Node Type 3.3.2.1. HPSS Server Nodes 3.3.2.1.1. AIX Requirements 3.3.2.1.2. Linux Requirements 3.3.2.2. HPSS Mover Nodes 3.3.2.2.1. AIX Requirements 3.3.2.2.2. Linux Requirements 3.3.2.2.3. Solaris Requirement s 3.3.2.2.4. IRIX Requirements 3.3.2.3. HPSS Client Nodes 3.3.2.3.1. SSM Client Requirem ents 3.3.2.3.2. Client API Require ments 3.4. Hardware Considerat ions 3.4.1. Network Consider ations 3.4.2. Robotically Mou nted Tape Page Table 2. Support ed Platform/Driver/Tap e Drive Combinations 3.4.4.1. Multiple Media Sup port Release 6.2 (Revision 2.0) 64 Page Table 3. Cartridg e/Drive Affinity Tabl e Release 6.2 (Revision 2.0) 66 3.4.5. Disk Devices 3.4.6. Special Bid Consid erations 3.5. HPSS Sizing Consider ations 3.5.1. HPSS User Storage Space 3.5.2. HPSS Infrastructu re Storage Space Page 3.5.3. HPSS Filesystems 3.5.3.1. /opt/hpss 3.5.3.2. /var/hpss 3.5.3.3. /var/hpss/ adm/core 3.5.3.4. /var/hpss/ hpssdb 3.5.3.5. /var/hpss/hpssdb/ subsys1 & subsysX 3.5.3.6. /db2/backup s/cfg Page 3.5.4. HPSS Metadata Spac e 3.5.4.1. SMS versus DMS S pace 3.5.4.2. 'CFG' Database Al location 3.5.4.3. 'SUBSYS' Database Allocation Page Page 3.5.4.4. DB2 Disk Space 3.5.5. System Memory and Disk Space 3.5.5.1. Operating Syste m Disk Spaces 3.5.5.2. System Disk Spa ce Requirements for Runnin g SSM 3.5.5.3. System Memory a nd Paging Space Requirements Table 4. Paging Space Info 3.6. HPSS Interface Co nsiderations Release 6.2 (Revision 2.0) 79 3.6.1. Client API 3.6.2. FTP 3.6.3. Parallel FTP 3.6.4. XFS 3.7. HPSS Server Considera tions 3.7.1. Core Server Page 3.7.2. Migration/Purge Se rver 3.7.3. Gatekeeper Page 3.7.4. Location Server 3.7.5. PVL 3.7.6. PVR 3.7.6.1. STK PVR 3.7.6.2. LTO PVR 3.7.6.3. 3494 PVR 3.7.6.4. AML PVR 3.7.6.5. Operator PVR 3.7.6.6. SCSI PVR 3.7.7. Mover 3.7.7.1. AIX Asynchronou s I/O 3.7.7.2. Tape Devices 3.7.7.2.1. AIX 3.7.7.2.2. Solaris 3.7.7.2.3. IRIX 3.7.7.2.4. Linux 3.7.7.3. Disk Devices 3.7.7.4. Performance 3.7.8. Logging Service 3.7.9. Startup Daemon 3.7.10. Storage System M anagement Key SM Environment Variables 3.8. Storage Subsystem Considerations 3.9. Storage Policy Co nsiderations 3.9.1. Migration Policy 3.9.1.1. Migration Poli cy for Disk 3.9.1.2. Migration Policy for Tape 3.9.2. Purge Policy 3.9.3. Accounting Policy and Validation Page 3.9.4. Security Policy 3.9.4.1. Client API 3.9.4.2. FTP/PFTP 3.9.4.3. XFS 3.9.4.4. Name Space 3.9.4.5. Security Audit 3.9.5. Logging Policy 3.9.6. Location Policy 3.9.7. Gatekeeping Table 5. Gatekee ping Call Parameters Release 6.2 (Revision 2.0) 100 3.10. Stor age Characteristics Cons iderations 3.10.1. Storage Class 3.10.1.1. Media Block Size Selection 3.10.1.2. Virtual Volume Blo ck Size Selection (disk ) 3.10.1.3. Virtual Volume Blo ck Size Selection (tape) 3.10.1.4. Stripe Width Selec tion 3.10.1.5. Blocks Between Ta pe Marks Selection (tape only) 3.10.1.6. Minimum Storage Seg ment Size Selection (disk only) 3.10.1.7. Maximum Storage Seg ment Size Selection (dis k only) 3.10.1.8. Maximum VVs to Wri te (tape only) 3.10.1.9. Average Number of S torage Segments (disk onl y) 3.10.1.10. PV Estimated Siz e / PV Size Selection 3.10.1.11. Optimum Access Size Selection 3.10.1.12. Some Recommended Pa rameter Values for Support ed Storage Media 3.10.1.12.1. Disk Media Paramete rs 3.10.1.12.2. Tape Media Parameter s Table 7. Sugges ted Block Sizes for Tape Release 6.2 (Revision 2.0) 108 3.10.2. Storage Hierarchy 3.10.3. Class of Service 3.10.3.1. Selecting Minim um File Size 3.10.3.2. Selecting Maximu m File Size 3.10.3.3. Selecting Stage Code 3.10.3.4. Selecting Optimum Access Size 3.10.3.5. Selecting Average Latency 3.10.3.6. Selecting Transfe r Rate 3.10.3.7. StripeLength and S tripeWidth Hints 3.10.4. File Families 3.11. HPSS Performance Conside rations 3.11.1. DB2 3.11.2. Bypassing Potenti al Bottlenecks 3.11.3. Configuration 3.11.4. FTP/PFTP 3.11.5. Client API 3.11.6. Core Server 3.11.7. Location Server 3.11.8. Logging 3.11.9. Cross Realm Trust 3.11.11. XFS 3.11.12. HPSS VFS Interface 3.12. HPSS Metadata Back up Considerations 3.13. HPSS Security Cons iderations Page Chapter 4. System Preparation 4.1. General Setup 4.2. Setup Filesystems 4.2.1. DB2 Filesystem 4.2.2. HPSS Filesystem 4.3. Setup Tape Libraries 4.3.1. Special LTO Conside rations 4.3.2. IBM 3584 4.3.3. 3494 4.3.4. STK 4.3.5. AML 4.4. Verify Tape Drives 4.4.1. AIX 4.4.2. Solaris 4.4.3. IRIX 4.4.4. Linux 4.5. Setup Disk Drives 4.5.1. AIX 4.5.2. Linux 4.5.3. IRIX 4.6. Setup Network Param eters Page Table 8. Network Options Release 6.2 (Revision 2.0) 131 4.6.1. HPSS.conf Configur ation File 4.6.2. SP/x Switch Devic e Buffer Driver Buffer Pools Page Chapter 5. HPSS Installation and Infrastructure Confi guration 5.1. Prepare for Installation 5.1.1. Distribution Media 5.1.2. Software Installation Packages Table 9. Install ation Package Sizes and Disk Requirements 5.1.3. Create Owner Accoun t for HPSS Files Release 6.2 (Revision 2.0) 136 5.1.4. Installation Targ et Directory Preparation 5.2. Install Prerequisi te Software 5.2.1. Install Java 5.2.2. Install MIT Ker beros (If Using Kerberos Authentication) 5.2.3. Install LDAP ( If Using LDAP Authorizatio n) 5.2.4. Install Prerequisi te Software for XFS HDM 5.3. Install HPSS/DB2 and Configure HPSS Infrast ructure 5.3.1. Install and Con figure HPSS - Root Subsyst em Machine 5.3.1.1. Pre-Installat ion Configuration Page 5.3.1.2. Install HPSS Documentation and DB2 Software 5.3.1.3. Set Up DB2 Permanent License 5.3.1.4. Configure HPSS Sec urity Services 5.3.1.4.1. Configure UNIX A uthentication and UNIX Au thorization Page Page 5.3.1.4.2. Configure Kerberos Authen tication and UNIX Authori zation Page Page 5.3.1.4.3. Configure Kerberos Authen tication and LDAP Author ization Page Page 5.3.1.5. Configure DB2 Services Page Page Page Page 5.3.1.5.1. Remote DB2 Client Ac cess & Fileset Creation/D eletion 5.3.1.6. Configure Other Se rvices 5.3.1.7. Create Configuration Bundle 5.3.2. Install and Config ure HPSS Secondary Sub system Machine 5.3.2.1. Pre-Installatio n Configuration Page 5.3.2.3. Set Up DB2 Permanent License 5.3.2.4. Install Configura tion Bundle 5.3.2.5. Configure HPSS Sec urity Services 5.3.2.6. Configure DB2 Services Page 5.3.2.7. Configure Other Services 5.3.3. Install and Configure HPSS Mover/Client Machine 5.3.3.1. Install Mover/Client sour ce code 5.3.3.2. Install Configura tion Bundle 5.3.3.3. Create /var/hpss subdirectories 5.3.3.4. Modify Kerberos Co nfiguration File, If Nec essary 5.3.3.5. Check Time and IP Address 5.4. Post Installation Pro cedures 5.5. HPSS Documentation & Manual Page Setup 5.5.1. Documentation and SSM Help Pac kage 5.5.2. Manual Page Setup 5.6. Define HPSS Environme nt Variables 5.7. Tune DB2 5.8. Install and Build HPSS Source Code 5.8.1. Construct and B uild the HPSS Base Source Tree 5.8.1.1. Construct th e HPSS Source Tree 5.8.1.2. Build the HPSS Base Source Tree 5.8.1.3. Generate and Bind the DB2 H elper Program 5.8.2. Construct and Build the HPSS Mover/Client Source Tree 5.8.2.1. Construct the HPSS Mover/ Client Source Tree 5.8.2.2. Build the HPSS Mover/Client Source Tree 5.8.3. Construct and Buil d the HPSS HDM Source Tree 5.8.3.1. Construct the H PSS HDM Source Tree 5.8.3.2. Build the HPSS H DM Source Tree 5.9. Supporting Both Unix and Kerberos Authenticati on for SSM Page Page Page Chapter 6. Upgrading to HPSS R elease 6.2 6.1. Special Instructions fo r Upgrading to HPSS 6.2.2 6.2. Planning for the HPSS 6.2 Upgrade 6.2.1. Metadata changes in HPSS 6.2 Page 6.2.2. Upgrade Requirements and Limitations 6.2.3. New Authentication and Authorization Mechan isms 6.2.3.1. Authentication Me chanisms 6.2.3.2. Authorization Mech anisms 6.2.4. New HPSS 6.2 System Files 6.2.5. Testing the Metada ta Conversion 6.2.6. Estimating the M etadata Conversion Time (fo r 4.5 upgrades only) 6.2.6.1. Running Time for th e Long Running Metadata Con version Utilities (for 4 .5 upgrades only) Table 10. Runin g Times for Long Runn ing Metadata Conversi on Utilities Release 6.2 (Revision 2.0) 185 6.2.7. Capturing the Metadat a Conversion Output 6.2.8. DB2 Configuration and Tuning (for 4.5 upgrade s only) Checking Database Configuration Settin gs Release 6.2 (Revision 2.0) 186 Updating the Data base Configuration 6.2.9. Overview of the U pgrade Utilities 6.2.9.1. HPSS 4.5 and HPSS 5.1 Upgrade Utilities Release 6.2 (Revision 2.0) 187 Authentication/Au thorization Upgrade Utilities Server Security AC L and Endpoint Creat ion Utilities 6.2.9.2. HPSS 4.5 Upgrade Ut ilities Release 6.2 (Revision 2.0) 188 HPSS 4.5 to 6.2 Metadata Conversion U tilities HPSS 4.5 to 6.2 Verification Utilities 6.2.9.3. HPSS 5.1 Upgrade Ut ilities Release 6.2 (Revision 2.0) 189 HPSS 5.1 to 6.2 Metadata Conversion U tilities HPSS 5.1 to 6.2 Verification Utilities 6.3. HPSS 6.2 Upgrade Pr ocedures Release 6.2 (Revision 2.0) 190 6.3.1. Verify Prerequis ites 6.3.2. Acquire Software 6.3.3. Install Authentic ation and Authorization Mechanisms Unix Authenticatio n and Authorization Release 6.2 (Revision 2.0) 191 Install and Confi gure Kerberos Install and Confi gure LDAP Page 6.3.4. Install or Upgrade DB2 Install and Confi gure DB2 (for 4.5 upg rades only) Release 6.2 (Revision 2.0) 193 Upgrading DB2 (for 5.1 upgrades only) 6.3.5. Upgrade AIX 6.3.6. Install or Upgrade Java 6.3.7. Save Current HPSS Code and Configuration Files 6.3.8. Prepare HPSS 6.2 C ode 6.3.8.1. Install HPSS 6.2 Distribution Image 6.3.8.2. Compile HPSS 6.2 Source Code (if necessary) 6.3.8.3. Disable Binaries, temporarily 6.3.9. Set Environment Vari ables Page Page 6.3.10. Setup Authenticat ion and Authorization HPSS Installation Guide July 2008 Release 6.2 (Revision 2.0) 200 Page 6.3.11. Pre-Conversion Sys tem Check 6.3.12. Take a full backup of SFS or DB2 6.3.13. Upgrade from HPSS 4 .5 to HPSS 6.2 6.3.13.1. Prepare for the Conversion 6.3.13.2. Run db_convert_c ollect_info to Collect Metadata Information 6.3.13.3. Convert Configurat ion Metadata 6.3.13.4. Convert Subsystem Metadata 6.3.13.5. Run the Long Running Utilities Page Restarting the Lo ng Running Utilities Release 6.2 (Revision 2.0) 207 Page 6.3.13.6. Create Core Server ACLs 6.3.13.7. Terminate the Scrip ting Session 6.3.13.8. Modify Permissions on Devices 6.3.14. Verify HPSS 4.5 C onversion Results 6.3.14.1. Capture Session Output 6.3.14.2. Run db_convert_s ize_check 6.3.14.3. Run db_convert_ns_ check 6.3.14.4. Run db_convert_add ress_check 6.3.14.5. Terminate Scripting Session 6.3.15. Upgrade from HPSS 5.1 to HPSS 6.2 Page 6.3.16. Enable DB2 Backup 6.3.17. Perform the DCE Exp ort: hpss_dce_export Release 6.2 (Revision 2.0) 215 6.3.18. Perform the Unix, LDAP or Kerberos Import Import DCE Authoriz ation Information in to Unix for HPSS Serv ers Import DCE Authoriz ation Information in to Unix for HPSS Users Import DCE Authoriz ation Information in to LDAP using hpss_ld ap_import Release 6.2 (Revision 2.0) 216 Create Local Site Information using hp ss_ldap_admin Import DCE Informat ion into Kerberos 6.3.19. Prepare the 6.2 Sy stem Page 6.3.19.8. Create SSM User Id s SSM User Ids under HPSS 4.5: Release 6.2 (Revision 2.0) 219 SSM User Ids under HPSS 5.1: Run hpssuser Utilit y 6.3.19.9. Create Location Se rver Endpoints 6.3.19.10. Perform Additiona l Remote Mover Configuratio n 6.3.20. Bring up the HPSS 6.2 Servers 6.3.20.1. Invoke the SSM Sy stem Manager, Startup Daem on and prerequisite soft ware Release 6.2 (Revision 2.0) 221 6.3.20.2. Update HPSS Configu rations Table 11. 6.2 D efault Server Configur ation Parameters Release 6.2 (Revision 2.0) 222 6.3.20.3. Dump Accounting Me tadata, if applicable 6.3.20.4. Start HPSS 6.2 Servers 6.3.21. Verify 6.2 System 6.3.22. Clean Up After a 4 .5 to 6.2 Upgrade 6.3.23. Clean Up After a 5 .1 to 6.2 Upgrade 6.3.24. Revert HPSS 6.2 S ystem to Prior Release 6.3.24.1. Revert the HPSS 6.2 System to Version 4. 5 6.3.24.2. Revert the HPSS 6.2 Syste m to Version 5.1 6.4. Metadata Conversion Troubleshooting Procedur es 6.4.1. HPSS 4.5 to 6.2 Conversion Utility Error s and Warnings 6.4.1.1. db_convert_co llect_info Errors 6.4.1.2. db_config_convert , db_subsys_convert, and db_lr_convert Errors and Warnings Page Page 6.4.2. HPSS 5.1 to 6.2 C onversion Utility Errors 6.4.2.1. hpss_md_convert _51 Errors 6.4.2.2. hpss_init_server _acls Errors Page 6.5.2. Examples of HPSS 4 .5 Conversion Utility O utput 6.5.2.1. db_convert_coll ect_info Output 6.5.2.2. db_config_conve rt Output Page Page Page 6.5.2.3. db_subsys_convert Output Page Page 6.5.2.4. Long Running Convers ion Utilities Output Page Page Page Appendix A. Glossary of Terms and Acronyms Page Page Page Page Page Page Page Page Page Page Appendix B. References Page Appendix C. Developer Acknowled gments Appendix D. HPSS.conf Configur ation File D.1. PFTP Client Stanza Release 6.2 (Revision 2.0) 259 Table 12. PFTP Client Stanza Fields Page Page Page Page D.2. PFTP Client Interf aces Stanza Table 13. PFTP Client Interfaces Stan za Fields Release 6.2 (Revision 2.0) 264 Page PFTP Client Interfaces Stanza Example: D.3. Multinode Table Sta nza Release 6.2 (Revision 2.0) 266 Table 14. Multin ode Table Stanza Fie lds Page D.4. Network Option Sta nza Table 15. Networ k Options Stanza Fiel ds Release 6.2 (Revision 2.0) 269 Page Page Name Section may be specified. NOTE: Do not include the quotes when specifying Default. D.5. PFTP Daemon Stanza Table 16. PFTP Daemon Stanza Descripti on Release 6.2 (Revision 2.0) 273 Page Page Page Page Page Page Page Page Page Page sizes in the range [1MB, 2MB). PFTP Daemon Stanza Example: Page Page D.6. Transfer Agent Sta nza Release 6.2 (Revision 2.0) 287 Table 17. Transf er Agent Stanza Descr iption Page Page Page D.7. Stanzas Reserved for Future Use Page Appendix E. hpss_env_defs.h Page Page Page Page Page Page Page Page Page Page Page Page Page Page }; Appendix F. /var/hpss files