March 2005, First Edition | IBM DS6000 Series specification

Summary of changes

This section describes the technical changes made in this edition of the book and in previous editions. This edition may also include minor corrections and editorial changes that are not identified.

Summary of Changes for SG24-6471-00

for DS6000 Series: Concepts and Architecture as created or updated on December 13, 2005.

March 2005, First Edition

This revision reflects the addition, deletion, or modification of new and changed information described below.

Changed information (see change bars)

Fixed errors in table on p. 330-331

SAN boot is available for the IBM® eServer BladeCenter®

Updated with August 2005 announcement information

Updated code load information

Updated DS CLI user management information

Updated DS CLI command examples with corrected syntax

Removed references to pre and post GA restrictions that no longer apply

Updated bibliography

© Copyright IBM Corp. 2004. All rights reserved.

xv

Image 17

IBM DS6000 Series manual March 2005, First Edition, Changed information see change bars

Contents

Architecture Page IBM TotalStorage DS6000 Series Concepts and Architecture First Edition March Iii Iv DS6000 Series Concepts and Architecture Contents IBM TotalStorage DS6000 model overview Part 3. Planning and configuration DS CLI Introduction Functionality Page Page Page Index Copyright License Trademarks Changed information see change bars March 2005, First Edition Xvi DS6000 Series Concepts and Architecture Xvii Team that wrote this redbook Xviii DS6000 Series Concepts and Architecture Preface Xx DS6000 Series Concepts and Architecture Comments welcome Become a published author Xxii DS6000 Series Concepts and Architecture Part 1 Introduction DS6000 Series Concepts and Architecture Introducing the IBM TotalStorage DS6000 series Infrastructure simplification DS6000 series, a member of the TotalStorage DS FamilyBusiness continuity Information lifecycle management Scaling options of the DS6000 and DS8000 series IBM TotalStorage DS6000 series unique benefits Processors Hardware overviewDS6800 controller enclosure Model Disk drives Switched FC-AL subsystemDense packaging Host adapters DS management console Storage capacityDS6000 expansion enclosure Model 1750-EX1 IBM TotalStorage DS Storage Manager Simulated offline configuration IBM TotalStorage DS Command-Line InterfaceReal-time online configuration Supported environment Business continuance functions Point-in-time Copy featureData Set FlashCopy Multiple Relationship FlashCopy Remote Mirror and Copy feature Global Copy Three sites solutionGlobal Mirror Remote Mirror connections Interoperability Service and setupResiliency Dynamic LUN/volume creation and deletion Configuration flexibilityLarge LUN and large CKD volume support Flexible LUN to LSS association Summary Positioning the IBM TotalStorage DS6000 seriesSimplified LUN masking DS6000 series compared to ESS Common set of functionsCommon management functions DS6000 series compared to DS4000 series DS6000 series compared to DS8000 series Performance Tagged Command QueuingUse with other virtualization products Self-learning cache algorithms Sarc IBM multipathing software Parallel Access Volumes PAVPerformance for zSeries Multiple Allegiance 20 DS6000 Series Concepts and Architecture Part 2 Architecture 22 DS6000 Series Concepts and Architecture Components Lower RAID controller Battery backup units Upper RAID controller Rear display panel Power supplyServer enclosure Front display panel Disk drive modules Expansion enclosure Upper Sbod controller Rear display panel Power supplyController architecture Lower Sbod controller SAN fabric Cache management Server-based design Randomseq Disk subsystem Non-switched FC-AL drawbacks Industry standard FC-AL disk enclosure Switched FC-AL advantages Disk enclosure DS6000 switched FC-AL implementation DDMs 10 Switched disk expansion Device adapter ports Server enclosure RAID controller cardTechnical details Host adapter port indicators Device adapter port indicatorsHost adapter ports SFPs Ethernet and serial ports Health indicatorsExpansion enclosure Sbod controller card Cabling Indicators 15 Expansion enclosure cabling Disk Contrl ports on loop Front panel Rear panel DS6000 front panel indicators DS6000 rear panel push buttons Enclosure ID indicatorRack identify connector 18 DS6000 power supplies Power subsystem DS6800 battery backup unit indicators Battery backup units Cables System service cardStorage Manager console Summary 44 DS6000 Series Concepts and Architecture RAS Data flow Controller RASFailover and failback DS6800 normal data flow Failover Failback NVS recovery after complete power loss Single and preferred path Host connection availabilityMetadata checks Controller SAN/FICON switches Multipathing software Open systems host connectionZSeries host connection RAID-5 overview Disk subsystem RASRAID-5 theory RAID-5 implementation in the DS6000 Drive failure RAID-10 overviewRAID-10 theory RAID-10 implementation in the DS6000 Floating spares Spare creation Disk scrubbing Predictive Failure Analysis PFADisk path redundancy Hot plugable DDMs DS6000 switched disk connections Power subsystem RAS Example 1 Using light path indicators to replace a DDM System serviceExample 2 Using the GUI to replace a power supply Redundant cooling Failed power supply Power supply replacement via the GUI System Identify Indicator blue light System indicatorsSystem Alert Indicator amber light System Information Indicator amber light Parts replacement Parts installation and repairsCRU parts versus FRU parts Tier 1 CRU parts Maintaining code Microcode updatesInstallation process Summary 64 DS6000 Series Concepts and Architecture Virtualization concepts Abstraction layers for disk virtualization Virtualization definition Physical layer as the base for virtualization Array sites Array sites Arrays Creation of an array Ranks Forming an FB rank with 1 GB extents Extent pools Server0 Fixed block LUNs Logical volumesCKD volumes Volume with ISeries LUNs Allocation and deletion of LUNs/CKD volumes Logical subsystems LSS DB2 Address groups Volume access Host attachment Volume group Summary of the virtualization hierarchy 10 Host attachments and volume groups 11 Virtualization hierarchy Placement of data 12 Optimal distribution of data Benefits of virtualization 82 DS6000 Series Concepts and Architecture IBM TotalStorage DS6000 model overview 1 DS6800 Model DS6000 highlights DS6800 Model 1750-511 rear view 2 DS6000 Model 1750-EX1 DS6800 Model 1750-EX1 rear view Designed to scale for capacity DS6800 physical capacity examples DS6800 capacity upgrade O L Copy Services Point-in-Time Copy FlashCopy Introduction to Copy ServicesCopy Services functions Read from the target volume Read from the source volumeWrite to the source volume Write to the target volume Refresh target volume also known as Incremental FlashCopy Point-in-Time Copy function authorizationFlashCopy options No background copy option Incremental FlashCopy Data Set FlashCopy Data Set FlashCopy Multiple Relationship FlashCopy Consistency Group FlashCopy What is Consistency Group FlashCopy? FlashCopy Source Volume FlashCopy Target Volume FlashCopy to Pprc Primary Volume Remote Mirror and Copy Peer-to-Peer Remote Copy Inband commands over remote mirror linkPersistent FlashCopy Metro Mirror Synchronous Pprc Metro Mirror Global Copy PPRC-XD Global Copy Global Mirror Asynchronous Pprc Global Mirror How Global Mirror works Global Mirror How it works OS Global Mirror XRC Ashynchronously Comparison of the Remote Mirror and Copy functions 104 DS6000 Series Concepts and Architecture What is data consistency? What is Consistency Group? 12 Consistency Group Example How does Consistency Group keep data consistency? LSS13 14 Consistency Group Example Interfaces for Copy Services DS Storage Manager Web-based interface DS Command-Line Interface CLI DS open application programming interface API Tip What is changed from the ESS CLI? Future Plan Interoperability with ESS 112 DS6000 Series Concepts and Architecture 113 Part 3 Planning and configuration 114 DS6000 Series Concepts and Architecture 115 Installation planning Floor and space requirements Installation site preparationGeneral considerations 117 Service clearance requirementsFloor load requirements Environmental requirements Power requirementsInput voltage requirements DS6000 series input voltage requirements Preparing the rack System management interfacesIBM TotalStorage DS Storage Manager 119 DS Open application programming interface DS Command-Line Interface 121 Network settings SAN environment SAN requirements and considerations 123 Attaching to an Open System hostFICON-attached zSeries Host Software requirements Licensed features 125 Configuration planning DS6000 Management Console Configuration planning considerations 127 Configuration management of DS6000 systemOffline configuration Express configuration Online configuration 129 Local maintenanceDS Management Console connectivity Remote service support Copy Services management Call home DS6000 licensed functionsSimple Network Management Protocol Snmp 131 License scope for each DS6000 licensed function Operating Environment License OEL required featureOperating environment license feature codes Point-in-Time Copy function PTC Optional featuresRemote Mirror and Copy functions RMC 133 Parallel Access Volume PAV feature codes Parallel Access Volumes PAVServer attachment license Ordering license functions User authorized to FlashCopy 25 TB of CKD data 135 Solution 137 Disk storage feature activation Capacity planning Physical configurations 139 Logical configurations CKD RAID rank capacity 141 Configuration with the same type of DDMsSparing rules Sparing Example 1 RAID 143 Configuration with different types of DDMs Sparing Example 3 RAID 145 Data migration planning Operating system mirroring Basic commandsSoftware packages Remote copy technologies 147 Migration appliances6 z/OS data migration methods Disk Magic Planning for performance Number of host ports Parallel Access Volumes z/OS onlyRemote copy 5 I/O priority queuing z/OS only Preferred paths Hot spot avoidance 151 DS Storage Manager Logical configuration Storage configuration terminology Configuration hierarchy, terminology, and conceptsStorage complex Storage unit PSeries2 Arrays Array sitesRanks Extent pools Extent Pool Volume groups Logical volumes Volume Group Address groups Example of Volume Group, LUNs and host attachment definition Extent Pool Copy services Relationships Raw DDMs PlanningArray Site Array layer Logical Configuration flow Logical volume layerRaw or physical DDM layer Array site layer Connecting to your DS6000 Introducing the GUI and logical configuration panels Introduction and Welcome panel 10 Entering the URL using the TCP/IP address for the SMC 12 View of the Welcome panel Real-time manager configuration Simulated manager configuration Copy Services Log Express configuration Create and define the users and passwords Using the help panels Software information center 17 View of the information center Navigating the GUI 19 View of the storage complexes in the work area 20 View of the Storage Complexes section Radio buttons and check boxes Configuring a storage complex Logical configuration process Configuring the storage unit 25 View of the General storage unit information panel Configuration advancements steps 28 Specify the I/O adapter configuration panel Configuring the logical host systems 30 View of Host Systems panel, with the Go button selected 32 View of Define Host Systems panel 34 View of the Select storage unit panel Click Apply assignment and OK Creating arrays from array sites 37View of the Array configuration Auto panel 38 View of the Add arrays to ranks panel, with FB selected 39 View of creating custom arrays from four disk array sites 40 View of the second array-site selection panel Creating extent pools 42 View of Define properties panel Creating FB volumes from extents Even numbered LSSs Radio button Click Volume Groups Creating volume groups 46 The Define volume group properties filled out Click Host Systems Deleting LUNs and recovering space in the extent poolAssigning LUNs to the hosts Creating CKD LCUs Creating CKD volumesClick Volumes → zSeries Click Express Configuration Wizard Using the Express Configuration Wizard 50 View of the Real-time Manager panel Click Storage units Wwnn number is displayed as shown in Figure 194 DS6000 Series Concepts and Architecture 195 Functionality Introduction Supported environments Installation methods ESS Copy Services command flow using ESS Copy Services CLI Command flow TCP/IP ports DS CLI command flowSecure sockets DS8000 split network DS6000 command flow ESS CLI co-existence Command flow for the DS6000 CLI co-existence Storage management Single command mode Command modesExample 10-1 Using DS CLI via a single command User security Script mode Interactive modeExample 10-4 Using DS CLI in interactive mode Example 10-3 Creating a DS CLI script User assistance Lists all available DS CLI commandsSyntax conventions Return codes Example 10-8 Use of the help -lcommandMan pages DS CLI return codes Example 10-10 Return code examples Example 10-11 Example of a configuration scriptUsage examples Mixed device environments and migration Which CLI to use based on what hardware you have installed Migration tasks DS CLI migration exampleDetermining the saved tasks to be migrated Migration considerations Collecting the task details A portion of the tasks listed by using the GUI Using the GUI to get the contents of a FlashCopy task Converting the saved task to a DS CLI command Converting a FlashCopy task to DS CLI Example 10-14 Using interactive dscli mode without profiles Creating a user ID for use only with ESS Using DS CLI commands via a single command or scriptProcedure to create an encrypted password file Example 10-15 Example of a dscli.profile file Setting up a profile Example 10-19 Creating an executable file Issuing a DS CLI command and scripting itExample 10-20 Using script mode 216 DS6000 Series Concepts and Architecture 217 218 DS6000 Series Concepts and Architecture 219 Performance considerations New and enhanced functions What is the challenge?Speed gap between server and disk storage 221 Where do we start? SSA backend interconnection Switch from Escon to Ficon portsArrays across loops Pprc over Fibre Channel links How does the DS6000 address the challenge? Fibre Channel switched disk interconnection at the back endFC-AL shortcomings How DS6000 series overcomes FC-AL shortcomings Fibre Channel switch 225 New four-port host adapters Fibre Channel device adapter 227 Enterprise-class dual cluster design for the DS6800 Power PC 229 Vertical growth and scalability Performance and sizing considerations for open systems Workload characteristics 231 Data placement in the DS6000LVM striping 10 Spreading data across ranks Stripe size Performance and sizing considerations for z/OS Determining where to attach the hostDetermining the number of paths to a LUN Subsystem Device Driver SDD Dynamic I/O load balancing Performance potential in z/OS environments Connect to zSeries hosts 235 An appropriate DS6000 size in z/OS environmentsProcessor memory size considerations for z/OS environments Disk array sizing considerations for z/OS environments Or zSeries channel consolidation 237 Configuration recommendations for z/OSConfigure one extent pool for each single rank Rank 239 Minimize the number of extent poolsPlan for a reasonable number of extent pools 14 Mix of extent pools 241 242 DS6000 Series Concepts and Architecture 243 ZSeries software enhancements Scalability support Software enhancements for the DS600012.2 z/OS enhancements Read availability mask support Large Volume Support LVSBenefits of the scalability enhancements 245 Read Control Unit and Device Recognition for DS6000 Initial Program Load IPL enhancements12.2.5 DS6000 definition to host software 247 New performance statisticsResource Measurement Facility RMF Preferred pathing Devserv Paths command showing preferred pathing 12.4 z/VSE enhancements 12.3 z/VM enhancementsMigration considerations Coexistence considerations TPF enhancements 251 Data Migration in zSeries environments Consolidate storage subsystems Consolidate logical volumesDefine migration objectives in z/OS environments 253 Keep source and target volume at the current sizeSummary of data migration objectives Physical migration with DFSMSdss and other storage software Data migration based on physical migration 255 Software- and hardware-based data migrationData migration with Piper for z/OS Piper hardware 257 Data migration with z/OS Global Mirror Bridge from Escon to Ficon with Metro/Global Copy Hardware- and microcode-based migration 259 Data migration with Metro Mirror or Global Copy 260 DS6000 Series Concepts and Architecture Pprcopy DDNAMEDD02 Query 261 If PENDING/SUSPEND Count of Tracks Remaining to be Copied = Example 13-2 All data is replicated Example 13-4 All data is replicated Data migration based on logical migration263 FDR Data Set Services Utility DFSMSdss SMS Storage Groups migration source environment Data migration within the system-managed storage environment Example 13-6 Select SMS storage group in Scds SG1 SG2 267 Volume Display, Define Alter or Delete Volume Information ALL Volumes Altered Example 13-10 Confirmation about SMS volume status change 269 Storgrp SG1 Combine physical and logical data migration 271 13.5 z/VM and VSE/ESA data migration Summary of data migration 273 274 DS6000 Series Concepts and Architecture 275 Open systems support and software Supported operating systems and servers Open systems support Where to look for updated and detailed information Platform and operating system vendors’ pages Differences to ESSQLogic Corporation Emulex Corporation Differences in interoperability between DS6000 and DS8000 Boot supportAdditional supported configurations RPQ Subsystem Device Driver DS CLI Other multipathing solutions IBM TotalStorage Productivity Center IBM TotalStorage Productivity Center MDM main panel Device Manager Sample Device Manager view TCP for Disk Example screenshot of TPC for Disk TPC for Replication Global Mirror Utility Enterprise Remote Copy Management Facility eRCMF 289 Data migration in the open Systems environment Few larger volumes 291 Comparison of migration methodsHost operating system-based migration Copy raw devices Basic copy commands 293 Migration using volume management softwareOnline copy and synchronization with rsync Migration using LVM mirroring Backup and Restore 295 Subsystem-based data migration Combination of Metro Mirror and Global Copy Metro Mirror and Global Copy 297 IBM Piper migration Other migration applications IBM migration services 299 Appendix A. Operating systems specifics DS6000 Host Systems Attachment Guide Data placement considerationsPlanning Capacity planning considerations Iostat Unix performance monitoring tools301 Example A-1 AIX iostat output Example A-2 SUN Solaris iostat output System Activity Report SAR Example A-3 SAR Sample Output 303 AIX host attachment scripts Other publicationsFinding the World Wide Port Names Example A-4 Finding Fibre Channel adapter WWN Useful SDD commands Determine the installed SDD levelManaging multiple paths Subsystem device driver SDD Multipath I/O Mpio Example A-7 lsvpcfg command Useful Mpio commands Determine the installed Sddpcm levelExample A-8 lspath command result 307 AIX access methods for I/O LVM configurationLVM striping LVM Mirroring AIX on IBM iSeries Boot device supportDirect I/O Concurrent I/O Iostat Monitoring I/O performanceFilemon Example A-9 Filemon output file 311 Linux Implementing Linux with IBM Disk Storage Existing reference materialLinux with zSeries and ESS Essentials Getting Started with zSeries Fibre Channel Protocol Table A-1 Major numbers and special device files Important Linux issuesSome Linux Scsi basics Example A-10 Create new special device files for Scsi disks Missing device files315 Scsi device assignment changes Managing multiple pathsLimited number of Scsi devices Fibre Channel disks discovered before internal Scsi disks RedHat Enterprise Linux RH-EL multiple LUN support317 Example A-12 Sample /etc/modules.conf Example A-13 SCSi disks attached at system start time Adding FC disks dynamically 319 Linux on IBM iSeriesGaps in the LUN sequence /proc pseudo file system Troubleshooting and monitoringExample A-16 Sample /proc/scsi/scsi file Example A-17 Sample /proc/scsi/qla2300/x Performance monitoring with iostat Microsoft Windows 2000/2003Generic Scsi tools 321 SDD for Windows HBA and operating system settings 323 Windows Server 2003 VDS support Volume Shadow Copy Service FC port configurationHP OpenVMS Geographically Dispersed Sites 325 Volume configuration OpenVMS volume shadowing Command Console LUN 327 328 DS6000 Series Concepts and Architecture 329 Appendix B. Using the DS6000 with iSeries Table B-1 OS/400 logical volume sizes Logical volume sizesSupported environment Hardware Changing LUN protection Protected versus unprotected volumes Using 5250 interface Adding volumes to iSeries configuration Figure B-2 Work with Disk Units menu Figure B-4 Specify ASPs to Add Units to Adding volumes to an Independent Auxiliary Storage Pool Figure B-6 iSeries Navigator initial panel Figure B-8 iSeries Navigator Disk Units Figure B-10 Create a new disk pool Figure B-12 Defining a new disk pool Figure B-14 Add disks to Disk Pool Figure B-16 Confirm disks to be added to Disk Pool Figure B-18 New Disk Pool Status Multipath Figure B-21 New logical volume shown on iSeries Navigator Figure B-22 Single points of failure Avoiding single points of failure Figure B-23 Multipath removes single points of failure Configuring multipath Figure B-24 Example of multipath with iSeries Adding multipath volumes to iSeries using 5250 interface Figure B-25 Adding multipath volumes to an ASP Adding volumes to iSeries using iSeries Navigator Figure B-27 Adding a multipath volume Figure B-28 New Disk Pool shown on iSeries Navigator Figure B-30 Example of multipath logical units Managing multipath volumes using iSeries Navigator 350 DS6000 Series Concepts and Architecture Figure B-32 Multipath logical unit properties Figure B-33 Multipath connections Multipath rules for multiple iSeries systems or partitions Preferred path for DS6000 Sizing guidelinesChanging from single path to multipath Cache Planning for arrays and DDMs Size and number of LUNs Number of iSeries Fibre Channel adaptersRecommended number of ranks Table B-2 Capacity per I/O Adapter Table B-3 Disk operations per second per RAID rank Connecting via SAN switchesSharing ranks between iSeries and other servers Metro Mirror and Global Copy MigrationOS/400 mirroring OS/400 data migration Figure B-36 Ending allocation for existing disk units Remote Mirror and Copy Copy Services for iSeriesFlashCopy AIX on IBM iSeries ISeries toolkit for Copy Services Linux on IBM iSeries 363 Appendix C. Service and support offerings IBM service offerings IBM Web sites for service offeringsIBM Implementation Services for TotalStorage disk systems IBM Implementation Services for TotalStorage Copy Functions Enterprise Remote Copy Management Facility eRCMF IBM Geographically Dispersed Parallel Sysplex Gdps IBM eServer iSeries Copy Services IBM Operational Support Services Support Line Appendix C. Service and support offerings 368 DS6000 Series Concepts and Architecture 369 IBM RedbooksOther publications Online resources Help from IBM How to get IBM Redbooks 372 DS6000 Series Concepts and Architecture 373 Index DS CLI ESS Interoperability with ESS Page Unix DS6000 Series Concepts and Architecture Page Page IBM TotalStorage DS6000 Series Concepts Architecture