Ess | IBM DS6000 Series guide

controller enclosure 6 controller RAS 46 Ethernet cables 42 FICON 50

host connection 49 interoperability 13 major features 7 microcode

installation process 62 maintaining 62

Model 1750-511 84 NVS recovery 48

open systems host connection 51 preferred path 49

SAN 50

server based design 27 server enclosure 24 service cable 42 system service card 42 tagged command queuing 18 zSeries host connection 51

dynamic LUN/volume creation and deletion 14

E

enclosure ID indicator 39

Enterprise Remote Copy Management Facility see eRC-

MF

environmental requirements 118 eRCMF 288, 365

ESS

interoperability with DS6000 111 ESS CLI 110

Ethernet cables 42 port 35

expansion enclosure 8, 25 cabling 36

express configuration 128, 167 using 191

wizard 165

extent pools 70, 154, 162 creating 184

F

FC-AL non-switched 29

FCP 122

Fibre Channel Protocol see FCP FICON 50, 122–123

filemon 310

fixed block LUNs 72 fixed block volumes 185 FlashCopy 90, 360

benefits 92

Consistency Group 95, 105 data sets 10

inband commands 11, 97 incremental 11

multiple relationship 10 options 92 persistent 97

floating spares 54

floor and space requirements 116 front panel 37

FRU 61

G

GDPS 365

GDS for MSCS 324, 366

Geographically Dispersed Sites for MSCS see GDS for

MSCS
Global Copy	12, 90, 98, 104, 259, 296, 357
Global Mirror	12, 90, 99, 104

Global Mirror Utility see GMU

GMU 287

H

HA 34 port

indicators 34 hardware components 23 hardware overview 6 host adapter see HA host attachment 78, 152

volume groups 78 host connection

open systems 51 zSeries 51

host systems 176 HP OpenVMS 324

command console LUN 326

FC port configuration 324 volume configuration 325 volume shadowing 326

I

I/O priority queuing 149 IASP 334

IBM Implementation Services CLI 364

Copy Functions 364 disk systems 364

IBM Migration Services 298 zSeries 364

IBM Migration services open systems 365

IBM Multi-path Subsystem Device Drive see SDD IBM TotalStorage DS Command-line Interface see DS CLI

IBM TotalStorage DS Storage Manager see DS Storage Manager

IBM TotalStorage Multiple Device Manager 283 IBM TotalStorage Productivity Center see TPC IBM TotalStorage SAN Volume Controller see SVC inband commands 11, 97

Index 375

Image 399

IBM DS6000 Series manual Ess

Contents

Architecture Page IBM TotalStorage DS6000 Series Concepts and Architecture First Edition March Iii Iv DS6000 Series Concepts and Architecture Contents IBM TotalStorage DS6000 model overview Part 3. Planning and configuration DS CLI Introduction Functionality Page Page Page Index Copyright License Trademarks Changed information see change bars March 2005, First Edition Xvi DS6000 Series Concepts and Architecture Xvii Team that wrote this redbook Xviii DS6000 Series Concepts and Architecture Preface Xx DS6000 Series Concepts and Architecture Comments welcome Become a published author Xxii DS6000 Series Concepts and Architecture Part 1 Introduction DS6000 Series Concepts and Architecture Introducing the IBM TotalStorage DS6000 series Information lifecycle management DS6000 series, a member of the TotalStorage DS FamilyInfrastructure simplification Business continuity Scaling options of the DS6000 and DS8000 series IBM TotalStorage DS6000 series unique benefits Hardware overview DS6800 controller enclosure ModelProcessors Host adapters Switched FC-AL subsystemDisk drives Dense packaging IBM TotalStorage DS Storage Manager Storage capacityDS management console DS6000 expansion enclosure Model 1750-EX1 Supported environment IBM TotalStorage DS Command-Line InterfaceSimulated offline configuration Real-time online configuration Multiple Relationship FlashCopy Point-in-time Copy featureBusiness continuance functions Data Set FlashCopy Remote Mirror and Copy feature Remote Mirror connections Three sites solutionGlobal Copy Global Mirror Service and setup ResiliencyInteroperability Flexible LUN to LSS association Configuration flexibilityDynamic LUN/volume creation and deletion Large LUN and large CKD volume support Positioning the IBM TotalStorage DS6000 series Simplified LUN maskingSummary Common set of functions Common management functionsDS6000 series compared to ESS DS6000 series compared to DS4000 series DS6000 series compared to DS8000 series Self-learning cache algorithms Sarc Tagged Command QueuingPerformance Use with other virtualization products Multiple Allegiance Parallel Access Volumes PAVIBM multipathing software Performance for zSeries 20 DS6000 Series Concepts and Architecture Part 2 Architecture 22 DS6000 Series Concepts and Architecture Components Front display panel Disk drive modules Upper RAID controller Rear display panel Power supplyLower RAID controller Battery backup units Server enclosure Lower Sbod controller Upper Sbod controller Rear display panel Power supplyExpansion enclosure Controller architecture SAN fabric Cache management Server-based design Randomseq Disk subsystem Non-switched FC-AL drawbacks Industry standard FC-AL disk enclosure Switched FC-AL advantages Disk enclosure DS6000 switched FC-AL implementation DDMs 10 Switched disk expansion Server enclosure RAID controller card Technical detailsDevice adapter ports SFPs Device adapter port indicatorsHost adapter port indicators Host adapter ports Health indicators Expansion enclosure Sbod controller cardEthernet and serial ports Cabling Indicators 15 Expansion enclosure cabling Disk Contrl ports on loop Front panel Rear panel DS6000 front panel indicators Enclosure ID indicator Rack identify connectorDS6000 rear panel push buttons 18 DS6000 power supplies Power subsystem DS6800 battery backup unit indicators Battery backup units System service card Storage Manager consoleCables Summary 44 DS6000 Series Concepts and Architecture RAS Controller RAS Failover and failbackData flow DS6800 normal data flow Failover Failback NVS recovery after complete power loss Host connection availability Metadata checksSingle and preferred path Controller SAN/FICON switches Open systems host connection ZSeries host connectionMultipathing software RAID-5 implementation in the DS6000 Disk subsystem RASRAID-5 overview RAID-5 theory RAID-10 implementation in the DS6000 RAID-10 overviewDrive failure RAID-10 theory Floating spares Spare creation Hot plugable DDMs Predictive Failure Analysis PFADisk scrubbing Disk path redundancy DS6000 switched disk connections Power subsystem RAS Redundant cooling System serviceExample 1 Using light path indicators to replace a DDM Example 2 Using the GUI to replace a power supply Failed power supply Power supply replacement via the GUI System Information Indicator amber light System indicatorsSystem Identify Indicator blue light System Alert Indicator amber light Tier 1 CRU parts Parts installation and repairsParts replacement CRU parts versus FRU parts Microcode updates Installation processMaintaining code Summary 64 DS6000 Series Concepts and Architecture Virtualization concepts Abstraction layers for disk virtualization Virtualization definition Physical layer as the base for virtualization Array sites Array sites Arrays Creation of an array Ranks Forming an FB rank with 1 GB extents Extent pools Server0 Logical volumes CKD volumesFixed block LUNs Volume with ISeries LUNs Allocation and deletion of LUNs/CKD volumes Logical subsystems LSS DB2 Address groups Volume access Host attachment Volume group Summary of the virtualization hierarchy 10 Host attachments and volume groups 11 Virtualization hierarchy Placement of data 12 Optimal distribution of data Benefits of virtualization 82 DS6000 Series Concepts and Architecture IBM TotalStorage DS6000 model overview 1 DS6800 Model DS6000 highlights DS6800 Model 1750-511 rear view 2 DS6000 Model 1750-EX1 DS6800 Model 1750-EX1 rear view Designed to scale for capacity DS6800 physical capacity examples DS6800 capacity upgrade O L Copy Services Introduction to Copy Services Copy Services functionsPoint-in-Time Copy FlashCopy Write to the target volume Read from the source volumeRead from the target volume Write to the source volume No background copy option Point-in-Time Copy function authorizationRefresh target volume also known as Incremental FlashCopy FlashCopy options Incremental FlashCopy Data Set FlashCopy Data Set FlashCopy Multiple Relationship FlashCopy Consistency Group FlashCopy What is Consistency Group FlashCopy? FlashCopy Source Volume FlashCopy Target Volume FlashCopy to Pprc Primary Volume Metro Mirror Synchronous Pprc Inband commands over remote mirror linkRemote Mirror and Copy Peer-to-Peer Remote Copy Persistent FlashCopy Metro Mirror Global Copy PPRC-XD Global Copy Global Mirror Asynchronous Pprc Global Mirror How Global Mirror works Global Mirror How it works OS Global Mirror XRC Ashynchronously Comparison of the Remote Mirror and Copy functions 104 DS6000 Series Concepts and Architecture What is data consistency? What is Consistency Group? 12 Consistency Group Example How does Consistency Group keep data consistency? LSS13 14 Consistency Group Example Interfaces for Copy Services DS Storage Manager Web-based interface DS Command-Line Interface CLI DS open application programming interface API Tip What is changed from the ESS CLI? Future Plan Interoperability with ESS 112 DS6000 Series Concepts and Architecture 113 Part 3 Planning and configuration 114 DS6000 Series Concepts and Architecture 115 Installation planning Installation site preparation General considerationsFloor and space requirements Service clearance requirements Floor load requirements117 DS6000 series input voltage requirements Power requirementsEnvironmental requirements Input voltage requirements 119 System management interfacesPreparing the rack IBM TotalStorage DS Storage Manager DS Open application programming interface DS Command-Line Interface 121 Network settings SAN environment SAN requirements and considerations Attaching to an Open System host FICON-attached zSeries Host123 Software requirements Licensed features 125 Configuration planning DS6000 Management Console Configuration planning considerations Configuration management of DS6000 system Offline configuration127 Express configuration Online configuration Local maintenance DS Management Console connectivity129 Remote service support Copy Services management 131 DS6000 licensed functionsCall home Simple Network Management Protocol Snmp Operating Environment License OEL required feature Operating environment license feature codesLicense scope for each DS6000 licensed function 133 Optional featuresPoint-in-Time Copy function PTC Remote Mirror and Copy functions RMC Ordering license functions Parallel Access Volumes PAVParallel Access Volume PAV feature codes Server attachment license User authorized to FlashCopy 25 TB of CKD data 135 Solution 137 Disk storage feature activation Capacity planning Physical configurations 139 Logical configurations CKD RAID rank capacity Configuration with the same type of DDMs Sparing rules141 Sparing Example 1 RAID 143 Configuration with different types of DDMs Sparing Example 3 RAID 145 Data migration planning Remote copy technologies Basic commandsOperating system mirroring Software packages Migration appliances 6 z/OS data migration methods147 Disk Magic Planning for performance 5 I/O priority queuing z/OS only Parallel Access Volumes z/OS onlyNumber of host ports Remote copy Preferred paths Hot spot avoidance 151 DS Storage Manager Logical configuration Storage unit Configuration hierarchy, terminology, and conceptsStorage configuration terminology Storage complex PSeries2 Extent pools Array sitesArrays Ranks Extent Pool Volume groups Logical volumes Volume Group Address groups Example of Volume Group, LUNs and host attachment definition Extent Pool Copy services Relationships Array layer PlanningRaw DDMs Array Site Array site layer Logical volume layerLogical Configuration flow Raw or physical DDM layer Connecting to your DS6000 Introducing the GUI and logical configuration panels Introduction and Welcome panel 10 Entering the URL using the TCP/IP address for the SMC 12 View of the Welcome panel Real-time manager configuration Simulated manager configuration Copy Services Log Express configuration Create and define the users and passwords Using the help panels Software information center 17 View of the information center Navigating the GUI 19 View of the storage complexes in the work area 20 View of the Storage Complexes section Radio buttons and check boxes Configuring a storage complex Logical configuration process Configuring the storage unit 25 View of the General storage unit information panel Configuration advancements steps 28 Specify the I/O adapter configuration panel Configuring the logical host systems 30 View of Host Systems panel, with the Go button selected 32 View of Define Host Systems panel 34 View of the Select storage unit panel Click Apply assignment and OK Creating arrays from array sites 37View of the Array configuration Auto panel 38 View of the Add arrays to ranks panel, with FB selected 39 View of creating custom arrays from four disk array sites 40 View of the second array-site selection panel Creating extent pools 42 View of Define properties panel Creating FB volumes from extents Even numbered LSSs Radio button Click Volume Groups Creating volume groups 46 The Define volume group properties filled out Deleting LUNs and recovering space in the extent pool Assigning LUNs to the hostsClick Host Systems Creating CKD volumes Click Volumes → zSeriesCreating CKD LCUs Click Express Configuration Wizard Using the Express Configuration Wizard 50 View of the Real-time Manager panel Click Storage units Wwnn number is displayed as shown in Figure 194 DS6000 Series Concepts and Architecture 195 Functionality Introduction Supported environments Installation methods ESS Copy Services command flow using ESS Copy Services CLI Command flow DS CLI command flow Secure socketsTCP/IP ports DS8000 split network DS6000 command flow ESS CLI co-existence Command flow for the DS6000 CLI co-existence Storage management User security Command modesSingle command mode Example 10-1 Using DS CLI via a single command Example 10-3 Creating a DS CLI script Interactive modeScript mode Example 10-4 Using DS CLI in interactive mode Lists all available DS CLI commands Syntax conventionsUser assistance DS CLI return codes Example 10-8 Use of the help -lcommandReturn codes Man pages Example 10-11 Example of a configuration script Usage examplesExample 10-10 Return code examples Mixed device environments and migration Which CLI to use based on what hardware you have installed Migration considerations DS CLI migration exampleMigration tasks Determining the saved tasks to be migrated Collecting the task details A portion of the tasks listed by using the GUI Using the GUI to get the contents of a FlashCopy task Converting the saved task to a DS CLI command Converting a FlashCopy task to DS CLI Example 10-14 Using interactive dscli mode without profiles Using DS CLI commands via a single command or script Procedure to create an encrypted password fileCreating a user ID for use only with ESS Example 10-15 Example of a dscli.profile file Setting up a profile Issuing a DS CLI command and scripting it Example 10-20 Using script modeExample 10-19 Creating an executable file 216 DS6000 Series Concepts and Architecture 217 218 DS6000 Series Concepts and Architecture 219 Performance considerations What is the challenge? Speed gap between server and disk storageNew and enhanced functions 221 Where do we start? Pprc over Fibre Channel links Switch from Escon to Ficon portsSSA backend interconnection Arrays across loops How DS6000 series overcomes FC-AL shortcomings Fibre Channel switched disk interconnection at the back endHow does the DS6000 address the challenge? FC-AL shortcomings Fibre Channel switch 225 New four-port host adapters Fibre Channel device adapter 227 Enterprise-class dual cluster design for the DS6800 Power PC 229 Vertical growth and scalability Performance and sizing considerations for open systems Workload characteristics Data placement in the DS6000 LVM striping231 10 Spreading data across ranks Stripe size Subsystem Device Driver SDD Dynamic I/O load balancing Determining where to attach the hostPerformance and sizing considerations for z/OS Determining the number of paths to a LUN Performance potential in z/OS environments Connect to zSeries hosts An appropriate DS6000 size in z/OS environments Processor memory size considerations for z/OS environments235 Disk array sizing considerations for z/OS environments Or zSeries channel consolidation Configuration recommendations for z/OS Configure one extent pool for each single rank237 Rank Minimize the number of extent pools Plan for a reasonable number of extent pools239 14 Mix of extent pools 241 242 DS6000 Series Concepts and Architecture 243 ZSeries software enhancements Software enhancements for the DS6000 12.2 z/OS enhancementsScalability support 245 Large Volume Support LVSRead availability mask support Benefits of the scalability enhancements Initial Program Load IPL enhancements 12.2.5 DS6000 definition to host softwareRead Control Unit and Device Recognition for DS6000 New performance statistics Resource Measurement Facility RMF247 Preferred pathing Devserv Paths command showing preferred pathing Coexistence considerations 12.3 z/VM enhancements12.4 z/VSE enhancements Migration considerations TPF enhancements 251 Data Migration in zSeries environments Consolidate logical volumes Define migration objectives in z/OS environmentsConsolidate storage subsystems Keep source and target volume at the current size Summary of data migration objectives253 Physical migration with DFSMSdss and other storage software Data migration based on physical migration Software- and hardware-based data migration Data migration with Piper for z/OS255 Piper hardware 257 Data migration with z/OS Global Mirror Bridge from Escon to Ficon with Metro/Global Copy Hardware- and microcode-based migration 259 Data migration with Metro Mirror or Global Copy 260 DS6000 Series Concepts and Architecture Pprcopy DDNAMEDD02 Query 261 If PENDING/SUSPEND Count of Tracks Remaining to be Copied = Example 13-2 All data is replicated Data migration based on logical migration 263Example 13-4 All data is replicated FDR Data Set Services Utility DFSMSdss SMS Storage Groups migration source environment Data migration within the system-managed storage environment Example 13-6 Select SMS storage group in Scds SG1 SG2 267 Volume Display, Define Alter or Delete Volume Information ALL Volumes Altered Example 13-10 Confirmation about SMS volume status change 269 Storgrp SG1 Combine physical and logical data migration 271 13.5 z/VM and VSE/ESA data migration Summary of data migration 273 274 DS6000 Series Concepts and Architecture 275 Open systems support and software Supported operating systems and servers Open systems support Where to look for updated and detailed information Emulex Corporation Differences to ESSPlatform and operating system vendors’ pages QLogic Corporation Boot support Additional supported configurations RPQDifferences in interoperability between DS6000 and DS8000 Subsystem Device Driver DS CLI Other multipathing solutions IBM TotalStorage Productivity Center IBM TotalStorage Productivity Center MDM main panel Device Manager Sample Device Manager view TCP for Disk Example screenshot of TPC for Disk TPC for Replication Global Mirror Utility Enterprise Remote Copy Management Facility eRCMF 289 Data migration in the open Systems environment Few larger volumes Comparison of migration methods Host operating system-based migration291 Copy raw devices Basic copy commands Migration using volume management software Online copy and synchronization with rsync293 Migration using LVM mirroring Backup and Restore 295 Subsystem-based data migration Combination of Metro Mirror and Global Copy Metro Mirror and Global Copy 297 IBM Piper migration Other migration applications IBM migration services 299 Appendix A. Operating systems specifics Capacity planning considerations Data placement considerationsDS6000 Host Systems Attachment Guide Planning Example A-1 AIX iostat output Unix performance monitoring toolsIostat 301 Example A-2 SUN Solaris iostat output System Activity Report SAR Example A-3 SAR Sample Output 303 Example A-4 Finding Fibre Channel adapter WWN Other publicationsAIX host attachment scripts Finding the World Wide Port Names Subsystem device driver SDD Determine the installed SDD levelUseful SDD commands Managing multiple paths Multipath I/O Mpio Example A-7 lsvpcfg command 307 Determine the installed Sddpcm levelUseful Mpio commands Example A-8 lspath command result LVM Mirroring LVM configurationAIX access methods for I/O LVM striping Concurrent I/O Boot device supportAIX on IBM iSeries Direct I/O Example A-9 Filemon output file Monitoring I/O performanceIostat Filemon 311 Linux Getting Started with zSeries Fibre Channel Protocol Existing reference materialImplementing Linux with IBM Disk Storage Linux with zSeries and ESS Essentials Important Linux issues Some Linux Scsi basicsTable A-1 Major numbers and special device files Missing device files 315Example A-10 Create new special device files for Scsi disks Managing multiple paths Limited number of Scsi devicesScsi device assignment changes Example A-12 Sample /etc/modules.conf RedHat Enterprise Linux RH-EL multiple LUN supportFibre Channel disks discovered before internal Scsi disks 317 Example A-13 SCSi disks attached at system start time Adding FC disks dynamically Linux on IBM iSeries Gaps in the LUN sequence319 Example A-17 Sample /proc/scsi/qla2300/x Troubleshooting and monitoring/proc pseudo file system Example A-16 Sample /proc/scsi/scsi file 321 Microsoft Windows 2000/2003Performance monitoring with iostat Generic Scsi tools SDD for Windows HBA and operating system settings 323 Windows Server 2003 VDS support Geographically Dispersed Sites FC port configurationVolume Shadow Copy Service HP OpenVMS 325 Volume configuration OpenVMS volume shadowing Command Console LUN 327 328 DS6000 Series Concepts and Architecture 329 Appendix B. Using the DS6000 with iSeries Hardware Logical volume sizesTable B-1 OS/400 logical volume sizes Supported environment Changing LUN protection Protected versus unprotected volumes Using 5250 interface Adding volumes to iSeries configuration Figure B-2 Work with Disk Units menu Figure B-4 Specify ASPs to Add Units to Adding volumes to an Independent Auxiliary Storage Pool Figure B-6 iSeries Navigator initial panel Figure B-8 iSeries Navigator Disk Units Figure B-10 Create a new disk pool Figure B-12 Defining a new disk pool Figure B-14 Add disks to Disk Pool Figure B-16 Confirm disks to be added to Disk Pool Figure B-18 New Disk Pool Status Multipath Figure B-21 New logical volume shown on iSeries Navigator Figure B-22 Single points of failure Avoiding single points of failure Figure B-23 Multipath removes single points of failure Configuring multipath Figure B-24 Example of multipath with iSeries Adding multipath volumes to iSeries using 5250 interface Figure B-25 Adding multipath volumes to an ASP Adding volumes to iSeries using iSeries Navigator Figure B-27 Adding a multipath volume Figure B-28 New Disk Pool shown on iSeries Navigator Figure B-30 Example of multipath logical units Managing multipath volumes using iSeries Navigator 350 DS6000 Series Concepts and Architecture Figure B-32 Multipath logical unit properties Figure B-33 Multipath connections Multipath rules for multiple iSeries systems or partitions Sizing guidelines Changing from single path to multipathPreferred path for DS6000 Cache Planning for arrays and DDMs Table B-2 Capacity per I/O Adapter Number of iSeries Fibre Channel adaptersSize and number of LUNs Recommended number of ranks Connecting via SAN switches Sharing ranks between iSeries and other serversTable B-3 Disk operations per second per RAID rank Migration OS/400 mirroringMetro Mirror and Global Copy OS/400 data migration Figure B-36 Ending allocation for existing disk units Copy Services for iSeries FlashCopyRemote Mirror and Copy AIX on IBM iSeries ISeries toolkit for Copy Services Linux on IBM iSeries 363 Appendix C. Service and support offerings IBM Implementation Services for TotalStorage Copy Functions IBM Web sites for service offeringsIBM service offerings IBM Implementation Services for TotalStorage disk systems Enterprise Remote Copy Management Facility eRCMF IBM Geographically Dispersed Parallel Sysplex Gdps IBM eServer iSeries Copy Services IBM Operational Support Services Support Line Appendix C. Service and support offerings 368 DS6000 Series Concepts and Architecture IBM Redbooks Other publications369 Online resources Help from IBM How to get IBM Redbooks 372 DS6000 Series Concepts and Architecture 373 Index DS CLI ESS Interoperability with ESS Page Unix DS6000 Series Concepts and Architecture Page Page IBM TotalStorage DS6000 Series Concepts Architecture