Power PC | IBM DS6000 Series specification

To host servers
			2 Gbps Fibre Channel ports
Adapter	Adapter			Controller
			Host adapter	Controller
			Host adapter	Card 1
			chipset	Card 1
			chipset
Processor	Processor	server	Power PC
		server	Power PC	Processor
		Storage	chipset	memory
Adapter	Adapter			Volatile
			Device adapter

				Persistent
			chipset	Persistent
			2 Gbps Fibre Channel ports

Figure 11-6 Standard PowerPC processor complexes for DS6800-511

The next figure, Figure 11-7,provides a less abstract view. It outlines some details on the dual processor complex of the DS6800 enclosure and its gates to host servers through HAs, and its connections to the disk storage backend through the DAs.

		To host servers
2 Gbps Fibre Channel ports					2 Gbps Fibre Channel ports
Host adapter	Controller	Adapter	Adapter	Controller
Host adapter	Card 0			Controller	Host adapter
chipset	Card 0			Card 1	Host adapter
chipset				Card 1	chipset
					chipset
Power PC	Processor	Processor	Processor
Power PC	Processor			Processor	Power PC
chipset	memory			Processor	Power PC
	memory			memory	chipset
				memory
	Volatile
	Volatile			Volatile
Device adapter		Adapter	Adapter	Volatile
Device adapter		Adapter	Adapter		Device adapter
chipset	Persistent				Device adapter
chipset	Persistent			Persistent	chipset
				Persistent	chipset
	2 Gbps Fibre Channel ports			2 Gbps Fibre Channel ports
				2 Gbps Fibre Channel ports
		Fibre Channel switch
			ooo
			16 DDM
		Fibre Channel switch

Figure 11-7 DS6800 server enclosure with its Fibre Channel switched disk subsystem

The DS6800 controls, through its two processor complexes, not only one I/O enclosure as Figure 11-7displays, but can connect to up to 7 expansion enclosures. Figure 11-8 on page 229 shows a DS6800 with one DS6000 expansion enclosure.

228DS6000 Series: Concepts and Architecture

Image 252

IBM DS6000 Series manual Power PC

Contents

Architecture Page IBM TotalStorage DS6000 Series Concepts and Architecture First Edition March Iii Iv DS6000 Series Concepts and Architecture Contents IBM TotalStorage DS6000 model overview Part 3. Planning and configuration DS CLI Introduction Functionality Page Page Page Index Copyright License Trademarks March 2005, First Edition Changed information see change bars Xvi DS6000 Series Concepts and Architecture Team that wrote this redbook Xvii Xviii DS6000 Series Concepts and Architecture Preface Xx DS6000 Series Concepts and Architecture Become a published author Comments welcome Xxii DS6000 Series Concepts and Architecture Part 1 Introduction DS6000 Series Concepts and Architecture Introducing the IBM TotalStorage DS6000 series DS6000 series, a member of the TotalStorage DS Family Infrastructure simplificationBusiness continuity Information lifecycle management IBM TotalStorage DS6000 series unique benefits Scaling options of the DS6000 and DS8000 series Hardware overview DS6800 controller enclosure ModelProcessors Switched FC-AL subsystem Disk drivesDense packaging Host adapters Storage capacity DS management consoleDS6000 expansion enclosure Model 1750-EX1 IBM TotalStorage DS Storage Manager IBM TotalStorage DS Command-Line Interface Simulated offline configurationReal-time online configuration Supported environment Point-in-time Copy feature Business continuance functionsData Set FlashCopy Multiple Relationship FlashCopy Remote Mirror and Copy feature Three sites solution Global CopyGlobal Mirror Remote Mirror connections Service and setup ResiliencyInteroperability Configuration flexibility Dynamic LUN/volume creation and deletionLarge LUN and large CKD volume support Flexible LUN to LSS association Positioning the IBM TotalStorage DS6000 series Simplified LUN maskingSummary Common set of functions Common management functionsDS6000 series compared to ESS DS6000 series compared to DS8000 series DS6000 series compared to DS4000 series Tagged Command Queuing PerformanceUse with other virtualization products Self-learning cache algorithms Sarc Parallel Access Volumes PAV IBM multipathing softwarePerformance for zSeries Multiple Allegiance 20 DS6000 Series Concepts and Architecture Part 2 Architecture 22 DS6000 Series Concepts and Architecture Components Upper RAID controller Rear display panel Power supply Lower RAID controller Battery backup unitsServer enclosure Front display panel Disk drive modules Upper Sbod controller Rear display panel Power supply Expansion enclosureController architecture Lower Sbod controller SAN fabric Server-based design Cache management Randomseq Non-switched FC-AL drawbacks Disk subsystem Switched FC-AL advantages Industry standard FC-AL disk enclosure DS6000 switched FC-AL implementation Disk enclosure 10 Switched disk expansion DDMs Server enclosure RAID controller card Technical detailsDevice adapter ports Device adapter port indicators Host adapter port indicatorsHost adapter ports SFPs Health indicators Expansion enclosure Sbod controller cardEthernet and serial ports Indicators Cabling Front panel 15 Expansion enclosure cabling Disk Contrl ports on loop DS6000 front panel indicators Rear panel Enclosure ID indicator Rack identify connectorDS6000 rear panel push buttons Power subsystem 18 DS6000 power supplies Battery backup units DS6800 battery backup unit indicators System service card Storage Manager consoleCables Summary 44 DS6000 Series Concepts and Architecture RAS Controller RAS Failover and failbackData flow Failover DS6800 normal data flow NVS recovery after complete power loss Failback Host connection availability Metadata checksSingle and preferred path SAN/FICON switches Controller Open systems host connection ZSeries host connectionMultipathing software Disk subsystem RAS RAID-5 overviewRAID-5 theory RAID-5 implementation in the DS6000 RAID-10 overview Drive failureRAID-10 theory RAID-10 implementation in the DS6000 Spare creation Floating spares Predictive Failure Analysis PFA Disk scrubbingDisk path redundancy Hot plugable DDMs Power subsystem RAS DS6000 switched disk connections System service Example 1 Using light path indicators to replace a DDMExample 2 Using the GUI to replace a power supply Redundant cooling Failed power supply Power supply replacement via the GUI System indicators System Identify Indicator blue lightSystem Alert Indicator amber light System Information Indicator amber light Parts installation and repairs Parts replacementCRU parts versus FRU parts Tier 1 CRU parts Microcode updates Installation processMaintaining code Summary 64 DS6000 Series Concepts and Architecture Virtualization concepts Virtualization definition Abstraction layers for disk virtualization Array sites Physical layer as the base for virtualization Arrays Array sites Ranks Creation of an array Extent pools Forming an FB rank with 1 GB extents Server0 Logical volumes CKD volumesFixed block LUNs Volume with Allocation and deletion of LUNs/CKD volumes ISeries LUNs Logical subsystems LSS DB2 Volume access Address groups Volume group Host attachment 10 Host attachments and volume groups Summary of the virtualization hierarchy Placement of data 11 Virtualization hierarchy Benefits of virtualization 12 Optimal distribution of data 82 DS6000 Series Concepts and Architecture IBM TotalStorage DS6000 model overview DS6000 highlights 1 DS6800 Model 2 DS6000 Model 1750-EX1 DS6800 Model 1750-511 rear view Designed to scale for capacity DS6800 Model 1750-EX1 rear view DS6800 capacity upgrade DS6800 physical capacity examples O L Copy Services Introduction to Copy Services Copy Services functionsPoint-in-Time Copy FlashCopy Read from the source volume Read from the target volumeWrite to the source volume Write to the target volume Point-in-Time Copy function authorization Refresh target volume also known as Incremental FlashCopyFlashCopy options No background copy option Data Set FlashCopy Incremental FlashCopy Multiple Relationship FlashCopy Data Set FlashCopy What is Consistency Group FlashCopy? Consistency Group FlashCopy FlashCopy to Pprc Primary Volume FlashCopy Source Volume FlashCopy Target Volume Inband commands over remote mirror link Remote Mirror and Copy Peer-to-Peer Remote CopyPersistent FlashCopy Metro Mirror Synchronous Pprc Global Copy PPRC-XD Metro Mirror Global Mirror Asynchronous Pprc Global Copy How Global Mirror works Global Mirror Global Mirror How it works OS Global Mirror XRC Comparison of the Remote Mirror and Copy functions Ashynchronously 104 DS6000 Series Concepts and Architecture What is Consistency Group? What is data consistency? How does Consistency Group keep data consistency? 12 Consistency Group Example LSS13 Interfaces for Copy Services 14 Consistency Group Example DS Command-Line Interface CLI DS Storage Manager Web-based interface Tip What is changed from the ESS CLI? DS open application programming interface API Interoperability with ESS Future Plan 112 DS6000 Series Concepts and Architecture Part 3 Planning and configuration 113 114 DS6000 Series Concepts and Architecture Installation planning 115 Installation site preparation General considerationsFloor and space requirements Service clearance requirements Floor load requirements117 Power requirements Environmental requirementsInput voltage requirements DS6000 series input voltage requirements System management interfaces Preparing the rackIBM TotalStorage DS Storage Manager 119 DS Command-Line Interface DS Open application programming interface Network settings 121 SAN requirements and considerations SAN environment Attaching to an Open System host FICON-attached zSeries Host123 Licensed features Software requirements Configuration planning 125 Configuration planning considerations DS6000 Management Console Configuration management of DS6000 system Offline configuration127 Online configuration Express configuration Local maintenance DS Management Console connectivity129 Copy Services management Remote service support DS6000 licensed functions Call homeSimple Network Management Protocol Snmp 131 Operating Environment License OEL required feature Operating environment license feature codesLicense scope for each DS6000 licensed function Optional features Point-in-Time Copy function PTCRemote Mirror and Copy functions RMC 133 Parallel Access Volumes PAV Parallel Access Volume PAV feature codesServer attachment license Ordering license functions 135 User authorized to FlashCopy 25 TB of CKD data Solution Disk storage feature activation 137 Physical configurations Capacity planning Logical configurations 139 CKD RAID rank capacity Configuration with the same type of DDMs Sparing rules141 Sparing Example 1 RAID Configuration with different types of DDMs 143 Sparing Example 3 RAID Data migration planning 145 Basic commands Operating system mirroringSoftware packages Remote copy technologies Migration appliances 6 z/OS data migration methods147 Planning for performance Disk Magic Parallel Access Volumes z/OS only Number of host portsRemote copy 5 I/O priority queuing z/OS only Hot spot avoidance Preferred paths DS Storage Manager Logical configuration 151 Configuration hierarchy, terminology, and concepts Storage configuration terminologyStorage complex Storage unit PSeries2 Array sites ArraysRanks Extent pools Extent Pool Logical volumes Volume groups Volume Group Example of Volume Group, LUNs and host attachment definition Address groups Extent Pool Copy services Relationships Planning Raw DDMsArray Site Array layer Logical volume layer Logical Configuration flowRaw or physical DDM layer Array site layer Introducing the GUI and logical configuration panels Connecting to your DS6000 10 Entering the URL using the TCP/IP address for the SMC Introduction and Welcome panel Real-time manager configuration 12 View of the Welcome panel Copy Services Simulated manager configuration Express configuration Log Using the help panels Software information center Create and define the users and passwords Navigating the GUI 17 View of the information center 19 View of the storage complexes in the work area Radio buttons and check boxes 20 View of the Storage Complexes section Logical configuration process Configuring a storage complex Configuring the storage unit 25 View of the General storage unit information panel Configuration advancements steps Configuring the logical host systems 28 Specify the I/O adapter configuration panel 30 View of Host Systems panel, with the Go button selected 32 View of Define Host Systems panel 34 View of the Select storage unit panel Creating arrays from array sites Click Apply assignment and OK 37View of the Array configuration Auto panel 38 View of the Add arrays to ranks panel, with FB selected 39 View of creating custom arrays from four disk array sites Creating extent pools 40 View of the second array-site selection panel Creating FB volumes from extents 42 View of Define properties panel Radio button Even numbered LSSs Creating volume groups Click Volume Groups 46 The Define volume group properties filled out Deleting LUNs and recovering space in the extent pool Assigning LUNs to the hostsClick Host Systems Creating CKD volumes Click Volumes → zSeriesCreating CKD LCUs Using the Express Configuration Wizard Click Express Configuration Wizard Click Storage units 50 View of the Real-time Manager panel Wwnn number is displayed as shown in Figure 194 DS6000 Series Concepts and Architecture 195 Introduction Functionality Installation methods Supported environments Command flow ESS Copy Services command flow using ESS Copy Services CLI DS CLI command flow Secure socketsTCP/IP ports DS6000 command flow DS8000 split network Command flow for the DS6000 ESS CLI co-existence Storage management CLI co-existence Command modes Single command modeExample 10-1 Using DS CLI via a single command User security Interactive mode Script modeExample 10-4 Using DS CLI in interactive mode Example 10-3 Creating a DS CLI script Lists all available DS CLI commands Syntax conventionsUser assistance Example 10-8 Use of the help -lcommand Return codesMan pages DS CLI return codes Example 10-11 Example of a configuration script Usage examplesExample 10-10 Return code examples Which CLI to use based on what hardware you have installed Mixed device environments and migration DS CLI migration example Migration tasksDetermining the saved tasks to be migrated Migration considerations A portion of the tasks listed by using the GUI Collecting the task details Converting the saved task to a DS CLI command Using the GUI to get the contents of a FlashCopy task Example 10-14 Using interactive dscli mode without profiles Converting a FlashCopy task to DS CLI Using DS CLI commands via a single command or script Procedure to create an encrypted password fileCreating a user ID for use only with ESS Setting up a profile Example 10-15 Example of a dscli.profile file Issuing a DS CLI command and scripting it Example 10-20 Using script modeExample 10-19 Creating an executable file 216 DS6000 Series Concepts and Architecture 217 218 DS6000 Series Concepts and Architecture Performance considerations 219 What is the challenge? Speed gap between server and disk storageNew and enhanced functions Where do we start? 221 Switch from Escon to Ficon ports SSA backend interconnectionArrays across loops Pprc over Fibre Channel links Fibre Channel switched disk interconnection at the back end How does the DS6000 address the challenge?FC-AL shortcomings How DS6000 series overcomes FC-AL shortcomings Fibre Channel switch 225 Fibre Channel device adapter New four-port host adapters Enterprise-class dual cluster design for the DS6800 227 Power PC Vertical growth and scalability 229 Workload characteristics Performance and sizing considerations for open systems Data placement in the DS6000 LVM striping231 Stripe size 10 Spreading data across ranks Determining where to attach the host Performance and sizing considerations for z/OSDetermining the number of paths to a LUN Subsystem Device Driver SDD Dynamic I/O load balancing Connect to zSeries hosts Performance potential in z/OS environments An appropriate DS6000 size in z/OS environments Processor memory size considerations for z/OS environments235 Or zSeries channel consolidation Disk array sizing considerations for z/OS environments Configuration recommendations for z/OS Configure one extent pool for each single rank237 Rank Minimize the number of extent pools Plan for a reasonable number of extent pools239 14 Mix of extent pools 241 242 DS6000 Series Concepts and Architecture ZSeries software enhancements 243 Software enhancements for the DS6000 12.2 z/OS enhancementsScalability support Large Volume Support LVS Read availability mask supportBenefits of the scalability enhancements 245 Initial Program Load IPL enhancements 12.2.5 DS6000 definition to host softwareRead Control Unit and Device Recognition for DS6000 New performance statistics Resource Measurement Facility RMF247 Devserv Paths command showing preferred pathing Preferred pathing 12.3 z/VM enhancements 12.4 z/VSE enhancementsMigration considerations Coexistence considerations TPF enhancements Data Migration in zSeries environments 251 Consolidate logical volumes Define migration objectives in z/OS environmentsConsolidate storage subsystems Keep source and target volume at the current size Summary of data migration objectives253 Data migration based on physical migration Physical migration with DFSMSdss and other storage software Software- and hardware-based data migration Data migration with Piper for z/OS255 Piper hardware Data migration with z/OS Global Mirror 257 Hardware- and microcode-based migration Bridge from Escon to Ficon with Metro/Global Copy Data migration with Metro Mirror or Global Copy 259 260 DS6000 Series Concepts and Architecture 261 Pprcopy DDNAMEDD02 Query Example 13-2 All data is replicated If PENDING/SUSPEND Count of Tracks Remaining to be Copied = Data migration based on logical migration 263Example 13-4 All data is replicated Data Set Services Utility DFSMSdss FDR Data migration within the system-managed storage environment SMS Storage Groups migration source environment SG1 SG2 Example 13-6 Select SMS storage group in Scds Volume Display, Define Alter or Delete Volume Information 267 Example 13-10 Confirmation about SMS volume status change ALL Volumes Altered 269 Combine physical and logical data migration Storgrp SG1 13.5 z/VM and VSE/ESA data migration 271 Summary of data migration 273 274 DS6000 Series Concepts and Architecture Open systems support and software 275 Open systems support Supported operating systems and servers Where to look for updated and detailed information Differences to ESS Platform and operating system vendors’ pagesQLogic Corporation Emulex Corporation Boot support Additional supported configurations RPQDifferences in interoperability between DS6000 and DS8000 Subsystem Device Driver Other multipathing solutions DS CLI IBM TotalStorage Productivity Center IBM TotalStorage Productivity Center Device Manager MDM main panel TCP for Disk Sample Device Manager view Example screenshot of TPC for Disk Global Mirror Utility TPC for Replication Enterprise Remote Copy Management Facility eRCMF Data migration in the open Systems environment 289 Few larger volumes Comparison of migration methods Host operating system-based migration291 Basic copy commands Copy raw devices Migration using volume management software Online copy and synchronization with rsync293 Backup and Restore Migration using LVM mirroring Subsystem-based data migration 295 Metro Mirror and Global Copy Combination of Metro Mirror and Global Copy IBM Piper migration 297 IBM migration services Other migration applications Appendix A. Operating systems specifics 299 Data placement considerations DS6000 Host Systems Attachment GuidePlanning Capacity planning considerations Unix performance monitoring tools Iostat301 Example A-1 AIX iostat output System Activity Report SAR Example A-2 SUN Solaris iostat output 303 Example A-3 SAR Sample Output Other publications AIX host attachment scriptsFinding the World Wide Port Names Example A-4 Finding Fibre Channel adapter WWN Determine the installed SDD level Useful SDD commandsManaging multiple paths Subsystem device driver SDD Example A-7 lsvpcfg command Multipath I/O Mpio Determine the installed Sddpcm level Useful Mpio commandsExample A-8 lspath command result 307 LVM configuration AIX access methods for I/OLVM striping LVM Mirroring Boot device support AIX on IBM iSeriesDirect I/O Concurrent I/O Monitoring I/O performance IostatFilemon Example A-9 Filemon output file 311 Linux Existing reference material Implementing Linux with IBM Disk StorageLinux with zSeries and ESS Essentials Getting Started with zSeries Fibre Channel Protocol Important Linux issues Some Linux Scsi basicsTable A-1 Major numbers and special device files Missing device files 315Example A-10 Create new special device files for Scsi disks Managing multiple paths Limited number of Scsi devicesScsi device assignment changes RedHat Enterprise Linux RH-EL multiple LUN support Fibre Channel disks discovered before internal Scsi disks317 Example A-12 Sample /etc/modules.conf Adding FC disks dynamically Example A-13 SCSi disks attached at system start time Linux on IBM iSeries Gaps in the LUN sequence319 Troubleshooting and monitoring /proc pseudo file systemExample A-16 Sample /proc/scsi/scsi file Example A-17 Sample /proc/scsi/qla2300/x Microsoft Windows 2000/2003 Performance monitoring with iostatGeneric Scsi tools 321 HBA and operating system settings SDD for Windows Windows Server 2003 VDS support 323 FC port configuration Volume Shadow Copy ServiceHP OpenVMS Geographically Dispersed Sites Volume configuration 325 Command Console LUN OpenVMS volume shadowing 327 328 DS6000 Series Concepts and Architecture Appendix B. Using the DS6000 with iSeries 329 Logical volume sizes Table B-1 OS/400 logical volume sizesSupported environment Hardware Protected versus unprotected volumes Changing LUN protection Adding volumes to iSeries configuration Using 5250 interface Figure B-2 Work with Disk Units menu Adding volumes to an Independent Auxiliary Storage Pool Figure B-4 Specify ASPs to Add Units to Figure B-6 iSeries Navigator initial panel Figure B-8 iSeries Navigator Disk Units Figure B-10 Create a new disk pool Figure B-12 Defining a new disk pool Figure B-14 Add disks to Disk Pool Figure B-16 Confirm disks to be added to Disk Pool Figure B-18 New Disk Pool Status Figure B-21 New logical volume shown on iSeries Navigator Multipath Avoiding single points of failure Figure B-22 Single points of failure Configuring multipath Figure B-23 Multipath removes single points of failure Adding multipath volumes to iSeries using 5250 interface Figure B-24 Example of multipath with iSeries Adding volumes to iSeries using iSeries Navigator Figure B-25 Adding multipath volumes to an ASP Figure B-27 Adding a multipath volume Figure B-28 New Disk Pool shown on iSeries Navigator Managing multipath volumes using iSeries Navigator Figure B-30 Example of multipath logical units 350 DS6000 Series Concepts and Architecture Figure B-32 Multipath logical unit properties Multipath rules for multiple iSeries systems or partitions Figure B-33 Multipath connections Sizing guidelines Changing from single path to multipathPreferred path for DS6000 Planning for arrays and DDMs Cache Number of iSeries Fibre Channel adapters Size and number of LUNsRecommended number of ranks Table B-2 Capacity per I/O Adapter Connecting via SAN switches Sharing ranks between iSeries and other serversTable B-3 Disk operations per second per RAID rank Migration OS/400 mirroringMetro Mirror and Global Copy OS/400 data migration Figure B-36 Ending allocation for existing disk units Copy Services for iSeries FlashCopyRemote Mirror and Copy ISeries toolkit for Copy Services AIX on IBM iSeries Linux on IBM iSeries Appendix C. Service and support offerings 363 IBM Web sites for service offerings IBM service offeringsIBM Implementation Services for TotalStorage disk systems IBM Implementation Services for TotalStorage Copy Functions IBM Geographically Dispersed Parallel Sysplex Gdps Enterprise Remote Copy Management Facility eRCMF IBM Operational Support Services Support Line IBM eServer iSeries Copy Services Appendix C. Service and support offerings 368 DS6000 Series Concepts and Architecture IBM Redbooks Other publications369 Online resources How to get IBM Redbooks Help from IBM 372 DS6000 Series Concepts and Architecture Index 373 DS CLI ESS Interoperability with ESS Page Unix DS6000 Series Concepts and Architecture Page Page IBM TotalStorage DS6000 Series Concepts Architecture