SAN requirements and considerations | IBM DS6000 Series instruction

7.5 SAN requirements and considerations

The DS6000 series provides a variety of host attachments so that you can consolidate storage capacity and workloads for open systems hosts and eServer zSeries hosts. The Fibre Channel adapters of the storage system can be configured to support the Fibre Channel Protocol (FCP) and fibre connection (FICON) protocol.

You should also keep the following consideration in mind: Fibre Channel SANs can provide the capability to interconnect open systems and storage in the same network as S/390 and zSeries host systems and storage.

Single Fibre Channel host adapters can have physical access to multiple Fibre Channel ports on the storage unit.

Part of the installation planning process is the planning for your SAN. You may already have a SAN infrastructure with enough unused ports to connect a DS6000, or you might need additional Fibre Channel switches. If you have switches that operate at 1 Gbps, you might consider replacing them with 2 Gbps switches. For high availability, you should connect the DS6000 at least to two different switches (see Figure 7-2).

The same considerations mentioned also apply to HBAs on the host. You might want to replace 1 Gbps HBAs by 2 Gbps HBAs. For fault tolerance, each server should be equipped with two HBAs connected to different Fibre Channel switches.

HBA-1

FC Switch 1

server0

HBA-2

FC Switch 1

server1

Fibre Channel SAN switches

DS6800

Figure 7-2 SAN environment

For Fibre Channel attachments, you can establish zones to limit the access of host adapters to storage system adapters. By establishing zones, you reduce the possibility of interactions between system adapters in switched configurations.

Part of the installation planning is to decide what SAN zones are needed or have to be modified.

122DS6000 Series: Concepts and Architecture

Image 146

IBM DS6000 Series manual SAN requirements and considerations, SAN environment

Contents

Architecture Page IBM TotalStorage DS6000 Series Concepts and Architecture First Edition March Iii Iv DS6000 Series Concepts and Architecture Contents IBM TotalStorage DS6000 model overview Part 3. Planning and configuration DS CLI Introduction Functionality Page Page Page Index Copyright License Trademarks March 2005, First Edition Changed information see change bars Xvi DS6000 Series Concepts and Architecture Team that wrote this redbook Xvii Xviii DS6000 Series Concepts and Architecture Preface Xx DS6000 Series Concepts and Architecture Become a published author Comments welcome Xxii DS6000 Series Concepts and Architecture Part 1 Introduction DS6000 Series Concepts and Architecture Introducing the IBM TotalStorage DS6000 series Business continuity DS6000 series, a member of the TotalStorage DS FamilyInfrastructure simplification Information lifecycle management IBM TotalStorage DS6000 series unique benefits Scaling options of the DS6000 and DS8000 series Processors Hardware overviewDS6800 controller enclosure Model Dense packaging Switched FC-AL subsystemDisk drives Host adapters DS6000 expansion enclosure Model 1750-EX1 Storage capacityDS management console IBM TotalStorage DS Storage Manager Real-time online configuration IBM TotalStorage DS Command-Line InterfaceSimulated offline configuration Supported environment Data Set FlashCopy Point-in-time Copy featureBusiness continuance functions Multiple Relationship FlashCopy Remote Mirror and Copy feature Global Mirror Three sites solutionGlobal Copy Remote Mirror connections Interoperability Service and setupResiliency Large LUN and large CKD volume support Configuration flexibilityDynamic LUN/volume creation and deletion Flexible LUN to LSS association Summary Positioning the IBM TotalStorage DS6000 seriesSimplified LUN masking DS6000 series compared to ESS Common set of functionsCommon management functions DS6000 series compared to DS8000 series DS6000 series compared to DS4000 series Use with other virtualization products Tagged Command QueuingPerformance Self-learning cache algorithms Sarc Performance for zSeries Parallel Access Volumes PAVIBM multipathing software Multiple Allegiance 20 DS6000 Series Concepts and Architecture Part 2 Architecture 22 DS6000 Series Concepts and Architecture Components Server enclosure Upper RAID controller Rear display panel Power supplyLower RAID controller Battery backup units Front display panel Disk drive modules Controller architecture Upper Sbod controller Rear display panel Power supplyExpansion enclosure Lower Sbod controller SAN fabric Server-based design Cache management Randomseq Non-switched FC-AL drawbacks Disk subsystem Switched FC-AL advantages Industry standard FC-AL disk enclosure DS6000 switched FC-AL implementation Disk enclosure 10 Switched disk expansion DDMs Device adapter ports Server enclosure RAID controller cardTechnical details Host adapter ports Device adapter port indicatorsHost adapter port indicators SFPs Ethernet and serial ports Health indicatorsExpansion enclosure Sbod controller card Indicators Cabling Front panel 15 Expansion enclosure cabling Disk Contrl ports on loop DS6000 front panel indicators Rear panel DS6000 rear panel push buttons Enclosure ID indicatorRack identify connector Power subsystem 18 DS6000 power supplies Battery backup units DS6800 battery backup unit indicators Cables System service cardStorage Manager console Summary 44 DS6000 Series Concepts and Architecture RAS Data flow Controller RASFailover and failback Failover DS6800 normal data flow NVS recovery after complete power loss Failback Single and preferred path Host connection availabilityMetadata checks SAN/FICON switches Controller Multipathing software Open systems host connectionZSeries host connection RAID-5 theory Disk subsystem RASRAID-5 overview RAID-5 implementation in the DS6000 RAID-10 theory RAID-10 overviewDrive failure RAID-10 implementation in the DS6000 Spare creation Floating spares Disk path redundancy Predictive Failure Analysis PFADisk scrubbing Hot plugable DDMs Power subsystem RAS DS6000 switched disk connections Example 2 Using the GUI to replace a power supply System serviceExample 1 Using light path indicators to replace a DDM Redundant cooling Failed power supply Power supply replacement via the GUI System Alert Indicator amber light System indicatorsSystem Identify Indicator blue light System Information Indicator amber light CRU parts versus FRU parts Parts installation and repairsParts replacement Tier 1 CRU parts Maintaining code Microcode updatesInstallation process Summary 64 DS6000 Series Concepts and Architecture Virtualization concepts Virtualization definition Abstraction layers for disk virtualization Array sites Physical layer as the base for virtualization Arrays Array sites Ranks Creation of an array Extent pools Forming an FB rank with 1 GB extents Server0 Fixed block LUNs Logical volumesCKD volumes Volume with Allocation and deletion of LUNs/CKD volumes ISeries LUNs Logical subsystems LSS DB2 Volume access Address groups Volume group Host attachment 10 Host attachments and volume groups Summary of the virtualization hierarchy Placement of data 11 Virtualization hierarchy Benefits of virtualization 12 Optimal distribution of data 82 DS6000 Series Concepts and Architecture IBM TotalStorage DS6000 model overview DS6000 highlights 1 DS6800 Model 2 DS6000 Model 1750-EX1 DS6800 Model 1750-511 rear view Designed to scale for capacity DS6800 Model 1750-EX1 rear view DS6800 capacity upgrade DS6800 physical capacity examples O L Copy Services Point-in-Time Copy FlashCopy Introduction to Copy ServicesCopy Services functions Write to the source volume Read from the source volumeRead from the target volume Write to the target volume FlashCopy options Point-in-Time Copy function authorizationRefresh target volume also known as Incremental FlashCopy No background copy option Data Set FlashCopy Incremental FlashCopy Multiple Relationship FlashCopy Data Set FlashCopy What is Consistency Group FlashCopy? Consistency Group FlashCopy FlashCopy to Pprc Primary Volume FlashCopy Source Volume FlashCopy Target Volume Persistent FlashCopy Inband commands over remote mirror linkRemote Mirror and Copy Peer-to-Peer Remote Copy Metro Mirror Synchronous Pprc Global Copy PPRC-XD Metro Mirror Global Mirror Asynchronous Pprc Global Copy How Global Mirror works Global Mirror Global Mirror How it works OS Global Mirror XRC Comparison of the Remote Mirror and Copy functions Ashynchronously 104 DS6000 Series Concepts and Architecture What is Consistency Group? What is data consistency? How does Consistency Group keep data consistency? 12 Consistency Group Example LSS13 Interfaces for Copy Services 14 Consistency Group Example DS Command-Line Interface CLI DS Storage Manager Web-based interface Tip What is changed from the ESS CLI? DS open application programming interface API Interoperability with ESS Future Plan 112 DS6000 Series Concepts and Architecture Part 3 Planning and configuration 113 114 DS6000 Series Concepts and Architecture Installation planning 115 Floor and space requirements Installation site preparationGeneral considerations 117 Service clearance requirementsFloor load requirements Input voltage requirements Power requirementsEnvironmental requirements DS6000 series input voltage requirements IBM TotalStorage DS Storage Manager System management interfacesPreparing the rack 119 DS Command-Line Interface DS Open application programming interface Network settings 121 SAN requirements and considerations SAN environment 123 Attaching to an Open System hostFICON-attached zSeries Host Licensed features Software requirements Configuration planning 125 Configuration planning considerations DS6000 Management Console 127 Configuration management of DS6000 systemOffline configuration Online configuration Express configuration 129 Local maintenanceDS Management Console connectivity Copy Services management Remote service support Simple Network Management Protocol Snmp DS6000 licensed functionsCall home 131 License scope for each DS6000 licensed function Operating Environment License OEL required featureOperating environment license feature codes Remote Mirror and Copy functions RMC Optional featuresPoint-in-Time Copy function PTC 133 Server attachment license Parallel Access Volumes PAVParallel Access Volume PAV feature codes Ordering license functions 135 User authorized to FlashCopy 25 TB of CKD data Solution Disk storage feature activation 137 Physical configurations Capacity planning Logical configurations 139 CKD RAID rank capacity 141 Configuration with the same type of DDMsSparing rules Sparing Example 1 RAID Configuration with different types of DDMs 143 Sparing Example 3 RAID Data migration planning 145 Software packages Basic commandsOperating system mirroring Remote copy technologies 147 Migration appliances6 z/OS data migration methods Planning for performance Disk Magic Remote copy Parallel Access Volumes z/OS onlyNumber of host ports 5 I/O priority queuing z/OS only Hot spot avoidance Preferred paths DS Storage Manager Logical configuration 151 Storage complex Configuration hierarchy, terminology, and conceptsStorage configuration terminology Storage unit PSeries2 Ranks Array sitesArrays Extent pools Extent Pool Logical volumes Volume groups Volume Group Example of Volume Group, LUNs and host attachment definition Address groups Extent Pool Copy services Relationships Array Site PlanningRaw DDMs Array layer Raw or physical DDM layer Logical volume layerLogical Configuration flow Array site layer Introducing the GUI and logical configuration panels Connecting to your DS6000 10 Entering the URL using the TCP/IP address for the SMC Introduction and Welcome panel Real-time manager configuration 12 View of the Welcome panel Copy Services Simulated manager configuration Express configuration Log Using the help panels Software information center Create and define the users and passwords Navigating the GUI 17 View of the information center 19 View of the storage complexes in the work area Radio buttons and check boxes 20 View of the Storage Complexes section Logical configuration process Configuring a storage complex Configuring the storage unit 25 View of the General storage unit information panel Configuration advancements steps Configuring the logical host systems 28 Specify the I/O adapter configuration panel 30 View of Host Systems panel, with the Go button selected 32 View of Define Host Systems panel 34 View of the Select storage unit panel Creating arrays from array sites Click Apply assignment and OK 37View of the Array configuration Auto panel 38 View of the Add arrays to ranks panel, with FB selected 39 View of creating custom arrays from four disk array sites Creating extent pools 40 View of the second array-site selection panel Creating FB volumes from extents 42 View of Define properties panel Radio button Even numbered LSSs Creating volume groups Click Volume Groups 46 The Define volume group properties filled out Click Host Systems Deleting LUNs and recovering space in the extent poolAssigning LUNs to the hosts Creating CKD LCUs Creating CKD volumesClick Volumes → zSeries Using the Express Configuration Wizard Click Express Configuration Wizard Click Storage units 50 View of the Real-time Manager panel Wwnn number is displayed as shown in Figure 194 DS6000 Series Concepts and Architecture 195 Introduction Functionality Installation methods Supported environments Command flow ESS Copy Services command flow using ESS Copy Services CLI TCP/IP ports DS CLI command flowSecure sockets DS6000 command flow DS8000 split network Command flow for the DS6000 ESS CLI co-existence Storage management CLI co-existence Example 10-1 Using DS CLI via a single command Command modesSingle command mode User security Example 10-4 Using DS CLI in interactive mode Interactive modeScript mode Example 10-3 Creating a DS CLI script User assistance Lists all available DS CLI commandsSyntax conventions Man pages Example 10-8 Use of the help -lcommandReturn codes DS CLI return codes Example 10-10 Return code examples Example 10-11 Example of a configuration scriptUsage examples Which CLI to use based on what hardware you have installed Mixed device environments and migration Determining the saved tasks to be migrated DS CLI migration exampleMigration tasks Migration considerations A portion of the tasks listed by using the GUI Collecting the task details Converting the saved task to a DS CLI command Using the GUI to get the contents of a FlashCopy task Example 10-14 Using interactive dscli mode without profiles Converting a FlashCopy task to DS CLI Creating a user ID for use only with ESS Using DS CLI commands via a single command or scriptProcedure to create an encrypted password file Setting up a profile Example 10-15 Example of a dscli.profile file Example 10-19 Creating an executable file Issuing a DS CLI command and scripting itExample 10-20 Using script mode 216 DS6000 Series Concepts and Architecture 217 218 DS6000 Series Concepts and Architecture Performance considerations 219 New and enhanced functions What is the challenge?Speed gap between server and disk storage Where do we start? 221 Arrays across loops Switch from Escon to Ficon portsSSA backend interconnection Pprc over Fibre Channel links FC-AL shortcomings Fibre Channel switched disk interconnection at the back endHow does the DS6000 address the challenge? How DS6000 series overcomes FC-AL shortcomings Fibre Channel switch 225 Fibre Channel device adapter New four-port host adapters Enterprise-class dual cluster design for the DS6800 227 Power PC Vertical growth and scalability 229 Workload characteristics Performance and sizing considerations for open systems 231 Data placement in the DS6000LVM striping Stripe size 10 Spreading data across ranks Determining the number of paths to a LUN Determining where to attach the hostPerformance and sizing considerations for z/OS Subsystem Device Driver SDD Dynamic I/O load balancing Connect to zSeries hosts Performance potential in z/OS environments 235 An appropriate DS6000 size in z/OS environmentsProcessor memory size considerations for z/OS environments Or zSeries channel consolidation Disk array sizing considerations for z/OS environments 237 Configuration recommendations for z/OSConfigure one extent pool for each single rank Rank 239 Minimize the number of extent poolsPlan for a reasonable number of extent pools 14 Mix of extent pools 241 242 DS6000 Series Concepts and Architecture ZSeries software enhancements 243 Scalability support Software enhancements for the DS600012.2 z/OS enhancements Benefits of the scalability enhancements Large Volume Support LVSRead availability mask support 245 Read Control Unit and Device Recognition for DS6000 Initial Program Load IPL enhancements12.2.5 DS6000 definition to host software 247 New performance statisticsResource Measurement Facility RMF Devserv Paths command showing preferred pathing Preferred pathing Migration considerations 12.3 z/VM enhancements12.4 z/VSE enhancements Coexistence considerations TPF enhancements Data Migration in zSeries environments 251 Consolidate storage subsystems Consolidate logical volumesDefine migration objectives in z/OS environments 253 Keep source and target volume at the current sizeSummary of data migration objectives Data migration based on physical migration Physical migration with DFSMSdss and other storage software 255 Software- and hardware-based data migrationData migration with Piper for z/OS Piper hardware Data migration with z/OS Global Mirror 257 Hardware- and microcode-based migration Bridge from Escon to Ficon with Metro/Global Copy Data migration with Metro Mirror or Global Copy 259 260 DS6000 Series Concepts and Architecture 261 Pprcopy DDNAMEDD02 Query Example 13-2 All data is replicated If PENDING/SUSPEND Count of Tracks Remaining to be Copied = Example 13-4 All data is replicated Data migration based on logical migration263 Data Set Services Utility DFSMSdss FDR Data migration within the system-managed storage environment SMS Storage Groups migration source environment SG1 SG2 Example 13-6 Select SMS storage group in Scds Volume Display, Define Alter or Delete Volume Information 267 Example 13-10 Confirmation about SMS volume status change ALL Volumes Altered 269 Combine physical and logical data migration Storgrp SG1 13.5 z/VM and VSE/ESA data migration 271 Summary of data migration 273 274 DS6000 Series Concepts and Architecture Open systems support and software 275 Open systems support Supported operating systems and servers Where to look for updated and detailed information QLogic Corporation Differences to ESSPlatform and operating system vendors’ pages Emulex Corporation Differences in interoperability between DS6000 and DS8000 Boot supportAdditional supported configurations RPQ Subsystem Device Driver Other multipathing solutions DS CLI IBM TotalStorage Productivity Center IBM TotalStorage Productivity Center Device Manager MDM main panel TCP for Disk Sample Device Manager view Example screenshot of TPC for Disk Global Mirror Utility TPC for Replication Enterprise Remote Copy Management Facility eRCMF Data migration in the open Systems environment 289 Few larger volumes 291 Comparison of migration methodsHost operating system-based migration Basic copy commands Copy raw devices 293 Migration using volume management softwareOnline copy and synchronization with rsync Backup and Restore Migration using LVM mirroring Subsystem-based data migration 295 Metro Mirror and Global Copy Combination of Metro Mirror and Global Copy IBM Piper migration 297 IBM migration services Other migration applications Appendix A. Operating systems specifics 299 Planning Data placement considerationsDS6000 Host Systems Attachment Guide Capacity planning considerations 301 Unix performance monitoring toolsIostat Example A-1 AIX iostat output System Activity Report SAR Example A-2 SUN Solaris iostat output 303 Example A-3 SAR Sample Output Finding the World Wide Port Names Other publicationsAIX host attachment scripts Example A-4 Finding Fibre Channel adapter WWN Managing multiple paths Determine the installed SDD levelUseful SDD commands Subsystem device driver SDD Example A-7 lsvpcfg command Multipath I/O Mpio Example A-8 lspath command result Determine the installed Sddpcm levelUseful Mpio commands 307 LVM striping LVM configurationAIX access methods for I/O LVM Mirroring Direct I/O Boot device supportAIX on IBM iSeries Concurrent I/O Filemon Monitoring I/O performanceIostat Example A-9 Filemon output file 311 Linux Linux with zSeries and ESS Essentials Existing reference materialImplementing Linux with IBM Disk Storage Getting Started with zSeries Fibre Channel Protocol Table A-1 Major numbers and special device files Important Linux issuesSome Linux Scsi basics Example A-10 Create new special device files for Scsi disks Missing device files315 Scsi device assignment changes Managing multiple pathsLimited number of Scsi devices 317 RedHat Enterprise Linux RH-EL multiple LUN supportFibre Channel disks discovered before internal Scsi disks Example A-12 Sample /etc/modules.conf Adding FC disks dynamically Example A-13 SCSi disks attached at system start time 319 Linux on IBM iSeriesGaps in the LUN sequence Example A-16 Sample /proc/scsi/scsi file Troubleshooting and monitoring/proc pseudo file system Example A-17 Sample /proc/scsi/qla2300/x Generic Scsi tools Microsoft Windows 2000/2003Performance monitoring with iostat 321 HBA and operating system settings SDD for Windows Windows Server 2003 VDS support 323 HP OpenVMS FC port configurationVolume Shadow Copy Service Geographically Dispersed Sites Volume configuration 325 Command Console LUN OpenVMS volume shadowing 327 328 DS6000 Series Concepts and Architecture Appendix B. Using the DS6000 with iSeries 329 Supported environment Logical volume sizesTable B-1 OS/400 logical volume sizes Hardware Protected versus unprotected volumes Changing LUN protection Adding volumes to iSeries configuration Using 5250 interface Figure B-2 Work with Disk Units menu Adding volumes to an Independent Auxiliary Storage Pool Figure B-4 Specify ASPs to Add Units to Figure B-6 iSeries Navigator initial panel Figure B-8 iSeries Navigator Disk Units Figure B-10 Create a new disk pool Figure B-12 Defining a new disk pool Figure B-14 Add disks to Disk Pool Figure B-16 Confirm disks to be added to Disk Pool Figure B-18 New Disk Pool Status Figure B-21 New logical volume shown on iSeries Navigator Multipath Avoiding single points of failure Figure B-22 Single points of failure Configuring multipath Figure B-23 Multipath removes single points of failure Adding multipath volumes to iSeries using 5250 interface Figure B-24 Example of multipath with iSeries Adding volumes to iSeries using iSeries Navigator Figure B-25 Adding multipath volumes to an ASP Figure B-27 Adding a multipath volume Figure B-28 New Disk Pool shown on iSeries Navigator Managing multipath volumes using iSeries Navigator Figure B-30 Example of multipath logical units 350 DS6000 Series Concepts and Architecture Figure B-32 Multipath logical unit properties Multipath rules for multiple iSeries systems or partitions Figure B-33 Multipath connections Preferred path for DS6000 Sizing guidelinesChanging from single path to multipath Planning for arrays and DDMs Cache Recommended number of ranks Number of iSeries Fibre Channel adaptersSize and number of LUNs Table B-2 Capacity per I/O Adapter Table B-3 Disk operations per second per RAID rank Connecting via SAN switchesSharing ranks between iSeries and other servers Metro Mirror and Global Copy MigrationOS/400 mirroring OS/400 data migration Figure B-36 Ending allocation for existing disk units Remote Mirror and Copy Copy Services for iSeriesFlashCopy ISeries toolkit for Copy Services AIX on IBM iSeries Linux on IBM iSeries Appendix C. Service and support offerings 363 IBM Implementation Services for TotalStorage disk systems IBM Web sites for service offeringsIBM service offerings IBM Implementation Services for TotalStorage Copy Functions IBM Geographically Dispersed Parallel Sysplex Gdps Enterprise Remote Copy Management Facility eRCMF IBM Operational Support Services Support Line IBM eServer iSeries Copy Services Appendix C. Service and support offerings 368 DS6000 Series Concepts and Architecture 369 IBM RedbooksOther publications Online resources How to get IBM Redbooks Help from IBM 372 DS6000 Series Concepts and Architecture Index 373 DS CLI ESS Interoperability with ESS Page Unix DS6000 Series Concepts and Architecture Page Page IBM TotalStorage DS6000 Series Concepts Architecture