Xviii DS8000 Series Concepts and Architecture | IBM manual

Front row - Cathy, Torsten R, Torsten K, Andre, Toni, Werner, Tetsuroh. Back row - Roland, Olivier, Anthony, Tang, Christine, Alex, Stu, Heinz, Chuck.

We want to thank all the members of John Amann’s team at the Washington Systems Center in Gaithersburg, MD for hosting us. Craig Gordon and Rosemary McCutchen were especially helpful in getting us access to beta code and hardware.

Thanks to the following people for their contributions to this project:

Susan Barrett

IBM Austin

James Cammarata

IBM Chicago

Dave Heggen

IBM Dallas

John Amann, Craig Gordon, Rosemary McCutchen

IBM Gaithersburg

Hartmut Bohnacker, Michael Eggloff, Matthias Gubitz, Ulrich Rendels, Jens Wissenbach,

Dietmar Zeller

IBM Germany

Brian Sherman

IBM Markham

Ray Koehler

IBM Minneapolis

John Staubi

IBM Poughkeepsie

Steve Grillo, Duikaruna Soepangkat, David Vaughn

IBM Raleigh

Amit Dave, Selwyn Dickey, Chuck Grimm, Nick Harris, Andy Kulich, Joe Prisco, Jim Tuckwell, Joe Writz

IBM Rochester

Charlie Burger, Gene Cullum, Michael Factor, Brian Kraemer, Ling Pong, Jeff Steffan, Pete Urbisci, Steve Van Gundy, Diane Williams

IBM San Jose

Jana Jamsek

IBM Slovenia

xviiiDS8000 Series: Concepts and Architecture

Image 20

IBM manual Xviii DS8000 Series Concepts and Architecture

Contents

IBM TotalStorage DS8000 Series Page IBM TotalStorage DS8000 Series Concepts and Architecture First Edition April Contents Page Benefits of virtualization Vi DS8000 Series Concepts and Architecture Summary Viii DS8000 Series Concepts and Architecture Page Page Index Xii DS8000 Series Concepts and Architecture Copyright License Trademarks Dfsort Preface Team that wrote this redbook Xvi DS8000 Series Concepts and Architecture Preface Xviii DS8000 Series Concepts and Architecture Become a published author Comments welcome Xx DS8000 Series Concepts and Architecture Part 1 Introduction DS8000 Series Concepts and Architecture Introduction to the DS8000 series DS8000, a member of the TotalStorage DS family Overview of the DS8000 seriesInfrastructure Simplification Business Continuity DS8000 Base frame Hardware overview Storage capacity Storage system logical partitions LPARsStorage Hardware Management Console S-HMC for the DS8000 IBM Standby Capacity on Demand offering for the DS8000 Supported environments Resiliency Family for Business Continuity IBM TotalStorage FlashCopy IBM TotalStorage Metro Mirror Synchronous Pprc Service and setup Three-site solutionInteroperability IBM TotalStorage Global Mirror Asynchronous Pprc DS8000 compared to ESS PositioningCommon set of functions IBM TotalStorage DS Command-Line Interface DS CLI Common management functionsDS8000 compared to DS6000 IBM TotalStorage DS Storage Manager DS Open application programming interface Scalability and configuration flexibilityFuture directions of storage system LPARs Sequential Prefetching in Adaptive Replacement Cache Sarc PerformanceIBM TotalStorage Multipath Subsystem Device Driver SDD Performance for zSeries Summary 16 DS8000 Series Concepts and Architecture Part 2 Architecture 18 DS8000 Series Concepts and Architecture Components Frames Base frame Expansion frame Rack operator panel Architecture Rack operator panel SAN fabric Server-based SMP design Cache management Components Processor complex Randomseq Components Service processor and Spcn Front viewRear view Processor memory RIO-G 2 I/O enclosures Device adapters Disk subsystemRear view Disk enclosures DS8000 device adapter Non-switched FC-AL drawbacks DS8000 disk enclosure Switched FC-AL advantages 11 DS8000 disk enclosure DS8000 switched FC-AL implementation 12 Disk enclosure switched connections Expansion Arrays and spares 14 DS8000 switched loop layout Arrays across loops DDMs Host adaptersAAL benefits Ficon and Fibre Channel protocol host adapters Control units and logical pathsEscon distances Remote Mirror and Copy with Escon Power and cooling Rack Power Control cards RPCFibre Channel supported servers Fibre Channel distances Primary power supplies Processor and I/O enclosure power suppliesDisk enclosure power and cooling Battery backup assemblies Ethernet switches 42 DS8000 Series Concepts and Architecture Storage system LPARs Logical partitions Introduction to logical partitioning Virtualization Engine technologyPartitioning concepts Partitions Logical partitioning Lpar Building blockPhysical partitioning Ppar Software and hardware fault isolation Dynamic logical partitioningMicro-Partitioning Virtual I/O Why Logically Partition? Server consolidationProduction and test environments Application isolation Increased flexibility of resource allocation DS8000 and LparLpar and storage facility images 2 DS8300 Lpar implementation DS8300 Model 9A2 Lpar and storage facility image Storage facility image hardware components DS8300 Lpar resource allocation Storage Facility Image Processor complex Processor and memory allocations 4 DS8300 Model 9A2 configuration options RIO-G interconnect separation DS8300 example configuration Model conversion Lpar security through Power Hypervisor Phyp Model conversions regarding Lpar functionality Lpar and Copy Services Lpar protection Power Hypervisor Remote mirroring Lpar benefitsFlashCopy Storage system LPARs Logical partitions Addressing capabilities with storage facility images Example of storage facility images in the DS8300 DS8300 addressing capabilities 60 DS8000 Series Concepts and Architecture RAS Naming Storage complexStorage unit Storage facility image Processor complex RAS Processor complex Reliability, availability, and serviceability Permanent monitoring +1 redundancy Memory reliability, fault tolerance, and integritySelf-healing Fault masking Concurrent MaintenanceHypervisor Storage image independence Resource deallocation Server RAS RIO-G a self-healing interconnect2 I/O enclosure Metadata checks Server failover and failback Data flow Failover Normal data flow NVS recovery after complete power loss Failback Host connection availability Single or multiple path Enclosure SAN/FICON/ESCON switches Multi-pathing software Open systems host connection ZSeries host connection Switched disk connections Disk path redundancy RAID-5 overview RAID-10 overviewRAID-5 theory RAID-5 implementation in the DS8000 RAID-10 implementation in the DS8000 Spare creationRAID-10 theory Hot plugable DDMs Predictive Failure Analysis PFAFloating spares Battery backup units Rack power control card RPCDisk scrubbing Rack cooling fans Building power loss Power fluctuation protectionPower control of the DS8000 Emergency power off EPO Microcode updates Installation process Different code versions across storage images Management consoleHMC considerations Virtualization concepts Virtualization definition Storage system virtualization Abstraction layers for disk virtualization Storage Facility virtualization Array sites Physical layer as the base for virtualization Arrays Array site Ranks Creation of an array Extent pools Forming an FB rank with 1 GB extents Server0 Fixed Block LUNs Logical volumesCKD volumes Volume with Allocation and deletion of LUNs/CKD volumes ISeries LUNs Logical subsystems LSS DB2 Volume access Address groups Volume group Host attachment 11 Host attachments and volume groups Summary of the virtualization hierarchy Placement of data 12 Virtualization hierarchy 13 Optimal placement of data Benefits of virtualization 101 102 DS8000 Series Concepts and Architecture IBM TotalStorage DS8000 model overview and scalability 103 DS8000 highlights Model naming conventions 2 DS8100 Model Maximum configuration for the Model 3 DS8300 Models 922 and 9A2 IBM TotalStorage DS8000 model overview and scalability Model comparison Designed for scalability Scalability for capacityLarge and scalable capacity DS8000 model comparison Scalability for performance Adding DDMsFuture plan Comparison of models for capacity Way I/O controllers Linear-scalable architecture RIO-G Benefit of the DS8000 for scalability Model upgrades Model conversions 114 DS8000 Series Concepts and Architecture Copy Services 115 Point-in-Time Copy FlashCopy Introduction to Copy ServicesCopy Services functions Write to the source volume Read from the source volumeRead from the target volume Point-in-Time Copy function authorization Refresh target volume also known as Incremental FlashCopyFlashCopy options No background copy option Data Set FlashCopy Incremental FlashCopy Multiple Relationship FlashCopy Data Set FlashCopy What is Consistency Group FlashCopy? Consistency Group FlashCopy FlashCopy to Pprc Primary Volume FlashCopy Source Volume FlashCopy Target Volume Inband Commands over Remote Mirror link Remote Mirror and Copy Peer-to-Peer Remote CopyPersistent FlashCopy Metro Mirror Synchronous Pprc Global Copy PPRC-XD Metro Mirror Global Mirror Asynchronous Pprc Global Copy How Global Mirror works Global Mirror Global Mirror How it works OS Global Mirror XRC Secondary server 11 z/OS Global Mirror Comparison of the Remote Mirror and Copy functions 12 z/OS Metro/Global Mirror Global Copy PPRC-XD What is a Consistency Group? What is data consistency? How does Consistency Group keep data consistency? 1st LSS13 Interfaces for Copy Services Storage Hardware Management Console S-HMC 16 DS8000 Copy Services network components DS Storage Manager Web-based interface DS Open application programming Interface API DS Command-Line Interface DS CLITip What is changed from the ESS CLI? Interoperability with ESS Future Plans 140 DS8000 Series Concepts and Architecture Part 3 Planning and configuration 141 142 DS8000 Series Concepts and Architecture Installation planning 143 Delivery requirements Packaged dimensions and weight for DS8000 modelsGeneral considerations 145 Installation site preparationFloor and space requirements Calculating space requirements Installing on raised or nonraised floorsMeeting floor load requirements Power requirements Power controlDS8000 dimensions 147 Power outlet requirements Input voltage requirements Power connector requirements Power consumption and environmental informationEnvironmental requirements Fans and air intake areas Cooling the storage complex Host attachmentAttaching to open systems hosts 151 ESCON-attached S/390 and zSeries hostsFICON-attached S/390 and zSeries hosts SAN Fabric products Where to get the updated information for host attachmentHost systems attachment Network and SAN requirements HMC network requirementsChannel extension technology products 153 SAN requirements Remote power control requirementsRemote support connection requirements 155 156 DS8000 Series Concepts and Architecture Configuration planning 157 Configuration planning overview Storage Hardware Management Console S-HMC S-HMC and Ethernet switches External S-HMC DS Storage Manager Remote servicesHMC software components 161 Required ports HMC network topology 163 Dial-up connectionSecure high-speed connection Remote access Call home HMC security considerations Security mechanism 1 Console must initiate sessionSecurity mechanism 2 Public key encryption 165 HMC user management Security mechanism 3 Login securityFTP Offload option Operating environment license OEL required feature Licensed function indicatorsDS8000 licensed functions 167 Optional features Operating environment license feature codesPoint-in-Time Copy function 2244 Model PTC License scope for each DS8000 licensed function Point-in-Time Copy PTC feature codes Remote Mirror and Copy RMC feature codesRemote Mirror and Copy functions 2244 Model RMC Remote Mirror for z/OS 2244 Model RMZ Parallel Access Volumes 2244 Model PAV Remote Mirror for zSeries RMZ feature codesParallel Access Volumes PAV feature codes Ordering licensed functions 171 User authorize to FlashCopy 25 TB of CKD data Solution Disk storage feature activation 173 Logical configurations Capacity planningScenarios for managing licensing Adding storage capacity to an existing licensed function 175 CKD RAID rank capacity Sparing rules FB RAID rank capacity Sparing examples 177 Sparing Example 2 RAID-10 179 11 Sparing example 3 RAID-5 Different capacity, same RPM Sparing Example 4 RAID-5 IBM Standby Capacity on Demand Standby CoD Capacity and well-balanced configuration 181 DDM to DA Mapping -- 2-way Data migration planning 183 Basic commands Operating system mirroringSoftware packages Remote copy technologies 185 Migration services and appliances6 z/OS data migration methods Planning for performance 15 Different data migration methods Parallel Access Volumes z/OS only Disk MagicSize of cache storage Number of host ports/channels Hot spot avoidance DS Storage Manager logical configuration 189 Configuration hierarchy, terminology, and concepts Storage configuration terminology PSeries1 PSeries2 Array sites ArraysRanks Extent pools Extent Pool Logical volumes Volume groups Volume Group Host System C Address groups Extent Pool Extent Pool Planning Array layer Raw or physical DDM layerArray site layer Logical Configuration flow Logical volume layerRank layer Extent pool layer Introducing the GUI and logical configuration panels Connecting to the DS8000 10 Entering the URL using the TCP/IP address for the S-HMC Welcome panel Real-time Manager configuration 12 The Welcome panel Copy Services Simulated Manager configuration Creating and defining the users and passwords Log Using the help panels information center 15 User administration panel Navigating the GUI 17 View of the information center 19 View of the storage complexes in the work area 20 View of the Storage Complexes section Radio buttons and check boxes 21 Storage unit view of the pull-down Logical configuration process Configuring a storage complex Configuring the storage unit 25 The General storage unit information panel Configuration advancement steps 28 Specify I/O adapter configuration panel Configuring the logical host systems 29 Create host systems, screen 31 View of the General host information panel 33 Define host ports panel, with updated host information Creating arrays from array sites Click Apply assignment and OK 36 The Definition method panel Creating extent pools 38 The Add arrays to ranks panel with FB selected Creating FB volumes from extents 39 The Definition method panel 42 The Define volume properties panel 41 The Select extent pool panel Creating volume groups Click Volume Groups 44 The Define volume group properties filled out Click Host Systems Deleting LUNs and recovering space in the extent poolAssigning LUNs to the hosts Creating CKD volumes Click Volumes → zSeriesCreating CKD LCUs Click Finish Displaying the storage unit Wwnn Click Storage Units 49 View of the Wwnn in the General panel 230 DS8000 Series Concepts and Architecture 231 Introduction Functionality Installation methods Supported environments Command flow ESS Copy Services command flow using ESS Copy Services CLI TCP/IP ports DS CLI command flowSecure sockets DS6000 command flow DS8000 split network Command flow for the DS6000 ESS CLI co-existence Storage management CLI co-existence Command modes Single command modeExample 11-1 Using DS CLI via a single command User security Interactive mode Script modeExample 11-4 Using DS CLI in interactive mode Example 11-3 Creating a DS CLI script Lists all available DS CLI commands Lists all DS CLI commands with syntax informationSyntax conventions User assistance Example 11-8 Use of the help -lcommand Return codesMan pages DS CLI return codes Example 11-10 Return code examples Example 11-11 Example of a configuration scriptUsage examples Which CLI to use based on what hardware you have installed Mixed device environments and migration DS CLI migration example Migration tasksDetermining the saved tasks to be migrated Migration considerations A portion of the tasks listed by using the GUI Collecting the task details Converting the saved task to a DS CLI command Using the GUI to get the contents of a FlashCopy task Example 11-14 Using interactive dscli mode without profiles Converting a FlashCopy task to DS CLI Using DS CLI commands via a single command or script Procedure to create an encrypted password fileSetting up a profile Creating a user ID for use only with ESS Issuing a DS CLI command and scripting it Example 11-18 Establishing a FlashCopy with a single command Example 11-20 Using script mode Example 11-19 Creating an executable file 252 DS8000 Series Concepts and Architecture Performance considerations 253 New and enhanced functions What is the challenge?Speed gap between server and disk storage Where do we start? 255 Switch from Escon to Ficon ports SSA backend interconnectionArrays across loops Pprc over Fibre Channel links Fibre Channel switched disk interconnection at the back end How does the DS8000 address the challenge?FC-AL shortcomings How the DS8000 series overcomes FC-AL shortcomings Memory 259 Fibre Channel device adapter New four-port host adapters POWER5 Heart of the DS8000 dual cluster design 261 RIO-G Interconnect RIO-G Interconnect 263 Performance and sizing considerations for open systems Vertical growth and scalability Workload characteristics Data placement in the DS8000Cache size considerations for open systems 265 LVM striping 266 DS8000 Series Concepts and Architecture Stripe size 267 Subsystem Device Driver SDD Dynamic I/O load-balancing Determining where to attach the hostDetermining the number of paths to a LUN 269 Performance and sizing considerations for z/OSConnect to zSeries hosts Performance potential in z/OS environments 12 DS8100 frontend connectivity example partial view 271 Appropriate DS8000 size in z/OS environmentsProcessor memory size considerations for z/OS environments Or zSeries channel consolidation Disk array sizing considerations for z/OS environments 273 Configuration recommendations for z/OS Configure one extent pool for each single rank Minimize the number of extent pools 275 Extent pool0 Extent pool1 Plan for a reasonable number of extent pools 277 15 Mix of extent pools 278 DS8000 Series Concepts and Architecture 279 280 DS8000 Series Concepts and Architecture ZSeries software enhancements 281 Scalability support Software enhancements for the DS800013.2 z/OS enhancements Large Volume Support LVS Read availability mask supportBenefits of the scalability enhancements 283 Read control unit and device recognition for DS8000 Initial Program Load IPL enhancements13.2.5 DS8000 definition to host software New performance statistics 285 Listdata Count 287 Listdata Counts report of DS8000 Listdata Status Resource Management Facility RMF 289 13.3 z/VM enhancements 13.4 z/VSE enhancementsMigration considerations Coexistence considerations TPF enhancements 291 292 DS8000 Series Concepts and Architecture Data migration in zSeries environments 293 Define migration objectives in z/OS environments Consolidate storage subsystems 295 Consolidate logical volumesConsiderations for new logical volume size Dynamic Parallel Access Volumes required for large volumes Example 14-1 Data Classes with EF attribute Keep source and target volume size at the current size 297 Physical migration with DFSMSdss and other storage software Data migration based on physical migrationSummary of data migration objectives 299 Software- and hardware-based data migrationData migration with Piper for z/OS Piper hardware Data migration with z/OS Global Mirror 301 Hardware- or microcode-based migration Bridge from Escon to Ficon with Metro/Global Copy Data migration with Metro Mirror or Global Copy 303 Pprcopy DDNAMEDD02 Query 305 Example 14-3 All data is replicated 491 Example 14-5 All data is replicated 307 Data Set Services UtilityData migration based on logical migration Hierarchical Storage Manager, DFSMShsm System utilities 309 SMS Storage Groups migration source environment Example 14-8 Select SMS storage group in Scds Storage Group Application Selection Alter Volume Statuses Pool Copy Pool Backup only 311 Example 14-12 Confirmation about SMS volume status change ALL Volumes Altered 313 Combine physical and logical data migration 315 14.5 z/VM and VSE/ESA data migrationSummary of data migration 316 DS8000 Series Concepts and Architecture 317 318 DS8000 Series Concepts and Architecture Open systems support and software 319 Supported operating systems and servers Where to look for updated and detailed informationOpen systems support DS8000 Interoperability Matrix IBM HBA Search ToolDS8000 Host Systems Attachment Guide TotalStorage Proven program Differences to the ESS Platform and operating system vendors’ pagesEmulex Corporation Atto Boot support Additional supported configurations RPQ Subsystem Device Driver Other multipathing solutions DS CLI IBM TotalStorage Productivity Center IBM TotalStorage Productivity Center MDM main panel Device Manager Sample Device Manager view TPC for Disk Sample screenshot of TPC for Disk Global Mirror Utility TPC for Replication Enterprise Remote Copy Management Facility eRCMF 332 DS8000 Series Concepts and Architecture Data migration in the open Systems environment 333 Tor Comparison of migration methods Host operating system-based migration Basic copy commands Copy raw devices Migration using volume management software Online copy and synchronization with rsync Backup and restore Migration using LVM mirroring Migration using backup and restore Subsystem-based data migration Metro Mirror and Global Copy Combination of Metro Mirror and Global Copy IBM Piper migration Piper migration IBM migration services Other migration applications Appendix A. Open systems operating systems specifics 343 Data placement considerations DS8000 Host Systems Attachment GuidePlanning Capacity planning considerations Example A-1 AIX iostat output Unix performance monitoring toolsIostat System Activity Report SAR Example A-2 SUN Solaris iostat output Vmstat Example A-3 SAR Sample Output Other publications AIX host attachment scriptsFinding the World Wide Port Names Example A-4 Finding Fibre Channel adapter WWN Determine the installed SDD level Useful SDD commandsManaging multiple paths Subsystem device driver SDD Example A-7 lsvpcfg command Multipath I/O Mpio Example A-8 lspath command result Determine the installed Sddpcm levelUseful Mpio commands LVM configuration AIX access methods for I/OLVM striping LVM mirroring Boot device support AIX on IBM iSeriesDirect I/O Concurrent I/O Monitoring I/O performance IostatFilemon Example A-9 Filemon output file IBM Mpio FC Linux Existing reference material Implementing Linux with IBM Disk StorageLinux with zSeries and ESS Essentials Getting Started with zSeries Fibre Channel Protocol Table A-1 Major numbers and special device files Important Linux issuesSome Linux Scsi basics Missing device files Example A-10 Create new special device files for Scsi disks Scsi device assignment changes Managing multiple pathsLimited number of Scsi devices RedHat Enterprise Linux RH-EL multiple LUN support Fibre Channel disks discovered before internal Scsi disks Example A-13 SCSi disks attached at system start time Adding FC disks dynamicallyExample A-12 Sample /etc/modules.conf Linux on IBM iSeries Gaps in the LUN sequence Example A-16 Sample /proc/scsi/scsi file Troubleshooting and monitoring/proc pseudo file system Example A-17 Sample /proc/scsi/qla2300/x Performance monitoring with iostatGeneric Scsi tools SDD for Windows HBA and operating system settingsMicrosoft Windows 2000/2003 Windows Server 2003 VDS support Figure A-1 Microsoft VDS Architecture FC port configuration Volume Shadow Copy ServiceHP OpenVMS Geographically Dispersed Sites Volume configuration Configurations Command Console LUN OpenVMS volume shadowing Appendix A. Open systems operating systems specifics 372 DS8000 Series Concepts and Architecture Appendix B. Using DS8000 with iSeries 373 Logical volume sizes Table B-1 OS/400 logical volume sizesSupported environment Hardware Protected versus unprotected volumes Changing LUN protection Adding volumes to iSeries configuration Using 5250 interface Figure B-3 Work with Disk Configuration menu Adding volumes to an Independent Auxiliary Storage Pool Figure B-5 Confirm Add Units Figure B-7 iSeries Navigator Signon to iSeries window Figure B-9 SST Signon Figure B-11 New disk pool welcome Figure B-13 Confirm disk pool configuration Figure B-15 Choose the disks to add to the Disk Pool Figure B-17 New Disk Pool Summary Figure B-20 New Disk Pool shown on iSeries Navigator Multipath Avoiding single points of failure Configuring multipath Figure B-22 Single points of failure Adding multipath volumes to iSeries using 5250 interface Figure B-24 Example of multipath with iSeries Figure B-25 Adding multipath volumes to an ASP Adding volumes to iSeries using iSeries Navigator Figure B-27 Adding a multipath volume Figure B-28 New Disk Pool shown on iSeries Navigator Managing multipath volumes using iSeries Navigator Figure B-30 Example of multipath logical units Appendix B. Using DS8000 with iSeries Figure B-32 Multipath logical unit properties Multipath rules for multiple iSeries systems or partitions Figure B-33 Multipath connections Sizing guidelines Changing from single path to multipath Planning for arrays and DDMs Cache Table B-2 Capacity per I/O Adapter Number of iSeries Fibre Channel adaptersSize and number of LUNs Table B-3 Disk operations per second per RAID rank Recommended number of ranksSharing ranks between iSeries and other servers Connecting via SAN switches MigrationOS/400 mirroring Metro Mirror and Global Copy Figure B-35 Using Metro Mirror to migrate from ESS to DS8000 OS/400 data migration Figure B-36 Ending allocation for existing disk units Remote Mirror and Copy Copy Services for iSeriesFlashCopy ISeries toolkit for Copy Services AIX on IBM iSeries Linux on IBM iSeries 406 DS8000 Series Concepts and Architecture Appendix C. Service and support offerings 407 IBM Web sites for service offerings IBM service offeringsIBM Implementation Services for TotalStorage disk systems IBM Implementation Services for TotalStorage Copy Functions IBM Geographically Dispersed Parallel Sysplex Gdps Enterprise Remote Copy Management Facility eRCMF IBM Operational Support Services Support Line IBM eServer iSeries Copy Services Appendix C. Service and support offerings 412 DS8000 Series Concepts and Architecture 413 IBM RedbooksOther publications Online resources How to get IBM Redbooks Help from IBM 416 DS8000 Series Concepts and Architecture Index 417 DS CLI OEL Escon Ficon RH-EL BBU Spcn VSE IBM TotalStorage DS8000 Series Concepts and Architecture Page Page IBM TotalStorage DS8000 Series Concepts Architecture